Data Programming In Python

Titanium alloys always contain titanium (Ti) as the main element and a second alloying element. These alloys are much stronger and have higher corrosion resistance at higher operating temperatures compared to commercially pure titanium (Ti) and the addition of the aluminium alpha (α) phase stabilizer is responsible for the increase in corrosion resistance and strength (Wu, 2006).

Single-phase α-titanium alloys exhibit higher brittleness compared to the other counterparts which consist of beta (β) phase stabilizers. The other variations of titanium alloys consist of the near alpha (α), 2-phase alpha (α) + beta (β), near beta (β), beta (β), near gamma (γ) and gamma phase (γ) alloys. The function of the beta (β) phase stabilizers is to increase the ductility for heat treatment processes such as annealing by increasing the beta (β) phase formation likelihood (Kim, 1995).

The formation of these phases can be interpreted using the titanium aluminide (TiAl) phase diagram with the working temperatures and the compositions of aluminium (Al) content. Each variation of titanium alloys exhibit different microstructures based on their cooling methods from a solid strengthened solution at high temperatures and their properties vary with respect to mechanical performance tests such as tensile, fatigue creep and compressive tests (Kim, 1995). Some of these tests can also be done at elevated temperatures to test their performances at high temperatures for evaluation of their properties for high temperature applications (Wu, 2006).

The most commercially used variation of the titanium alloys is the titanium aluminide (TiAl) 2-phase alpha (α) + beta (β) alloy with aluminium (4%) as the second phase particle for strengthening the alpha (α) phase and vanadium (V) as the beta (β) phase stabilizer (6%) for strengthening the beta (β) phase and the name of the alloy is Ti-6Al-4V (Kim, 1995). The Ti-6Al-4V alloy is commonly used to manufacture compressor blades for turbine engines on the cold section of the engine where heat is not significantly present (Kim, 1995). Medical grade Ti-6Al-4V can be used for orthopaedic and dental applications for bone replacement (Rodriguez, Bowen, Zelzer and Stamboulis, 2020).

However, there are 2 variations of titanium aluminide (TiAl) alloys which are not commonly used and it is still a subject of research for properties determination and applications and that variation of titanium aluminide (TiAl) is the gamma phase (γ) alloy and the alpha (α2) phase (Ti3Al) (McBride, Malinov and Sha, 2004). The gamma phase (γ) can also be mixed with the (α2) phase (Ti3Al) ((α2) + (γ)) during microstructure formation upon cooling. Their respective formation temperatures with respect to the percentage of aluminium composition additions can be interpreted in the phase diagram just as with the other variations of titanium alloys. The 2-phase alpha (α) + beta (β) alloys are the ones used to currently manufacture the blades.

# Import required modules for image processing and plotting

import matplotlib.pyplot as plt
from PIL import Image

# Phase diagram illustration to show the various phases formed for the TiAl alloys with respect to temperature and Al% content

Phase_Diagram = Image.open(r"./Phase_Diagram.png")

plt.imshow(Phase_Diagram)

plt.show()

From the phase diagram displayed above, with the aluminium alloying additions increasing beyond 20% as indicated on the lower x-axis, the titanium alloy will form titanium aluminide intermetallic compounds (γ) (TiAl) and ((α2) + (γ)) (Ti3Al + TiAl) phases upon cooling from the liquid (L) phase due to the high weight percentages of aluminium atoms bonding with the titanium atoms in large quantities.

Several other compounds formed besides TiAl (γ), Ti3Al ((α2) are the TiAl2, TiAl3 and α-TiAl3 (Ma et al., 2020). The 2 dominant phases that make up the overall varying compositions of various (γ) (TiAl) and ((α2) + (γ)) (Ti3Al + TiAl) titanium aluminides are the TiAl (γ), Ti3Al ((α2) phases with aluminium atomic percentages above 20%.

1.2 Titanium Aluminide (TiAl) Dataset Origins, Reproduction and Permissions for Reusability¶

Back To Top

The titanium aluminide (TiAl) dataset being used for this coursework is part of an actual research project conducted by Nanyang Technological University (NTU) and Institute of High-Performance Computing (IHPC) in Singapore to determine several predicted material properties that can be used to synthesize a new titanium aluminide (TiAl) alloy by incorporating several existing properties into artificial neural networks.

This dataset was compiled in 2019 by me and 2 other researchers at the Nanyang Technological University (NTU). The final expected outcome of the actual project will be to have an artificial neural network with a good predictive model to predict several combinations of the best material properties for the (γ) and ((α2) + (γ)) titanium alumimides which can rival the properties of nickel superalloys for use in the hot sections of turbine engines in terms of strength, durability and long-term fatigue with the use of artificial neural networks.

This dataset is useful as it includes a total of 284 types of existing titanium aluminide (TiAl) alloys with a wide range of essential material properties like second element alloying compositions, cost, yield strength, tensile strength, elongation, fracture toughness, processing methods etc. This dataset is not readily available on the internet as it is confidential in terms of free sharing but special permission was granted by the project managers at both abovementioned institutes to make use of this dataset for this coursework.

This dataset was also chosen as it has been used by fellow data scientists who are involved in this project and it has also been incorporated into Alchemite which is an artificial neural network made by a company known as Intellegens in the UK for cost predictions as well as property predictions. Intellegens (UK) is also collaborating with Nanyang Technological University (NTU) and the Institute of High-Performance Computing (IHPC) for this project and the chief technical officer Dr Gareth Conduit is responsible for training researchers on the fundamentals and aspects of using neural networks such as Alchemite as a machine learning method to predict material properties.

1.3 Materials Science, Machine Learning and Artificial Intelligence¶

Back To Top

Artificial intelligence has now become advanced enough that it can applied to materials science. This is achievable by compiling experimental data results inputs with complex mathematical permutations for correlation through a user-defined ANN with specific sets of programmed flow of instructions and predict the probability of synthesizing a new material with the desired newly tailored properties for the desired applications (Malinov and Sha, 2004).

To understand the capabilities and mechanical properties of (γ) and ((α2) + (γ)), artificial intelligence can be deployed using an ANN where a set of programmable instructions can be made using a programmable language such as python with experimental data inputs being administered with a sequential flow of mathematical calculations and iterations. This will then determine if the predicted material properties can fit the criteria of being used in the hot sections of aircraft turbine engines in conjunction with nickel superalloys or as a replacement for nickel superalloys.

With the combination of the appropriate input and output parameters related to the composition-property and property-property relationships and training methods, it is possible to create an ANN that can predict the material properties which are necessary for the (γ) and ((α2) + (γ)) to have for them to operate at high temperature conditions in aircraft turbine engines.

1.4 Aims and Objectives¶

Back To Top

In coursework 1, a linear regression model was used to predict the cost values using the atomic percentage composition (at%) values for each element present within each variation of the (γ) and ((α2) + (γ)) titanium aluminide (TiAl) alloy and the output R2 value was compared with Alchemite which was the artificial neural network that was used for the actual project. This was the aim and objective of coursework 1.

It was seen that the best prediction made by the linear regression model (R2 = 0.9989) and Alchemite (R2 = 0.9751) were very similar, indicating that the linear regression model can predict the cost values efficiently. A K-Fold cross validation was also done to display the respective R2 values for each segment of the dataset with 5 splits with 80% of the data being used as the training set and 20% of the data being used as the test set and the respective R2 values were within the range of 0.80 to 0.99 with everchanging seed values. The noisy version of the data provided lower R2 values with larger deviations of mean squared errors.

The same linear regression model however, may not be used to predict the essential mechanical properties such as the yield strength and tensile strength as the predictions can possibly be no longer linear, leading to possible non-linear relationships between the independent (X) and dependent (y) (yield strength, tensile strength, Elongation (Log 10)) variables.

For this coursework, the essential mechanical properties which are the yield strength, tensile strength and Elongation (Log 10) shall be predicted based on the multi-layer perceptron (MLP) neural network model and the R2 values for each of the predictions of each of the mechanical properties will be compared with the respective R2 values provided by Alchemite to distinguish the advantages and disadvantages seen between both neural networks.

By comparing the R2 values between both neural networks and their respective advantages and disadvantages, it will contribute a huge step towards the primary outcome of the actual project stated below. The cost predictions will also be done using the multi-layer perceptron (MLP) neural network model and the R2 value derived will be compared with the R2 value derived from the linear regression model to verify if it can reproduce the same results as the linear regression model.

The primary expected outcome of the actual project will be to have an artificial neural network with a good predictive model to predict several combinations of the best material properties for the (γ) and ((α2) + (γ)) titanium alumimides which can rival the properties of nickel superalloys for use in the hot sections of turbine engines in terms of strength, durability and long-term fatigue.

2. Data Preprocessing and Preparation¶

2.1 Modules Import¶

Back To Top

The modules required for this coursework were imported accordingly from various python libraries. The modules necessary for image plotting and display have been defined earlier as the titanium aluminide (TiAl) phase diagram had to be explained as part of the introduction and the image had to be imported into python earlier.

# Import required modules 

import numpy as np
import pandas as pd
import statistics as st
import matplotlib
import matplotlib.pyplot as plt
import re as re
import string
import itertools
import statsmodels.api as sm

from PIL import Image 
from nltk.stem.snowball import SnowballStemmer
from nltk.util import ngrams
from statistics import mean
from numpy import arange
from scipy.stats import pearsonr

from sklearn.preprocessing import LabelEncoder, StandardScaler
from sklearn.metrics import mean_squared_error, r2_score
from sklearn.model_selection import train_test_split, KFold, cross_val_score, RandomizedSearchCV
from sklearn.linear_model import LinearRegression 
from sklearn.neural_network import MLPRegressor

2.2 Dataframe Settings Modification¶

Back To Top

The dataframe dimensions had to be maximized to accomodate all dimensions of the dataframe without truncation to allow the data within the large dataframes to be visualised easily, hence the dataframe maximizing settings were initialized.

# Maximize dataframe rows and columns for easier visualisations

pd.set_option('display.max_rows', None)
pd.set_option('display.max_columns', None)
pd.set_option('display.max_colwidth', None)
pd.set_option('display.width', None)
pd.set_option('display.expand_frame_repr', True)

2.3 Dataset Import¶

Back To Top

The dataset was first imported as an excel spreadsheet.

# Import dataset (excel)

TiAl_Input = pd.read_excel(r"./Dataset2.xlsx")

# Convert dataset (csv)

TiAl_Input.to_csv(r"./Dataset2.csv", index=None, header=True)

2.4 Dataset Conversion¶

Back To Top

Following the import of the dataset as an excel spreadsheet, a conversion process was done to convert it from the excel workbook format (.xlsx) to a comma seperated values (.csv) format for allow working with the data to be much easier.

# Read and display dataset (csv)

TiAl_Input = pd.read_csv(r"./Dataset2.csv")

TiAl_Input

2.5 Remove Unnecessary Columns¶

Back To Top

The next step is to remove unnecessary columns to preserve only the data columns that will be used for the prediction of the cost, yield strength, tensile strength and elongation. In this coursework, the input prarameters that will be used as the independent (X) variable will be the composition atomic percentages (at%) columns, the synthesis methods column and the microstructure column. One of the dependent variable (y) for prediction will be the cost column which has not been defined in the dataframe yet as well as the yield strength, tensile strength and elongation columns.

The cost data to be used for the training and test sets must be calculated using the elements weight percentages (wt%) and the individual elements cost values. As the elements weight percentages (wt%) and cost values have not been calculated, they must be calculated and added into the dataframe.

The columns that have many missing values and those not essential to the calculation of the cost input for training and testing and predicted cost output for now, have been removed accordingly.

# Drop unnessary columns

TiAl_Input_drop = TiAl_Input.drop(["Alloy composition (At %)  Dec 2019 1228", "Elongation (%)",
                                   "Fracture Toughness (MPa)", "Primary Creep (Elongation/s)", 
                                   "Secondary Creep (Elongation/s)", "Lamaller Volume", "Interlamellar Spacing (µm)",
                                   "α Width (µm)", "γ Width (µm)", "γ Grain Size (µm)", "α2-α2 Spacing (µm)", "Vα", "Vγ",
                                   "Colony Size (µm)", "Density (g/cm³)", "Diamond Pyramid Hardness (DPH)", 
                                   "Vickers Hardness (HV)", "Conditions", 
                                   "Orientation (°)"], 
                                   axis=1)

TiAl_Input_drop

2.6 Replace Missing Values¶

Back To Top

As the independent (X) variables will be the compositions atomic percentages (at%) columns the synthesis methods column and the microstructure column, they need to be preserved, cleaned and manipulated accordingly. All elements compositions atomic percentages (at%) columns where missing values were present were filled with zeros to create a sparse dataframe with the assumption that if there is any missing value for any specific element for any specific alloy, that element is not present in that specific alloy and that particular information regarding the missing value of that element composition atomic percentage (at%) can be made to be zero.

# Fill missing values of composition columns with zeros to create sparse data along missing areas

TiAl_Input_drop.loc[:, "Ti(at%)": "TiB2(at%)"] = TiAl_Input_drop.loc[:, "Ti(at%)": "TiB2(at%)"].fillna(0)

TiAl_Input_drop

2.7 Convert Elements Atomic Percentages (at%) to Weight Percentages (wt%)¶

Back To Top

After filling the missing values of the elements compositions atomic percentages (at%) columns with zeros to create a sparse dataframe, the weight percentages (wt%) of each element was calculated using each element's atomic number as it is neccesary to use the elements compositions weight percentages (wt%) to calculate the cost which will be one of the target variables (y) that will be split into testing and training datasets with the independent (X) variables.

The weight percentages (wt%) of each element was calculated by multiplying the individual respective atomic number of each element by the respective dataframe containing all their atomic percentages (at%), divided by the sum of all multiplied individual respective atomic numbers of all elements with their respective dataframes containing all their atomic percentages (at%).

# Coversion of elements compositions atomic percentages (at%) to weight percentages (%) using each element's atomic number

Ti_Element_Weight_Percentages = TiAl_Input_drop["Ti(at%)"]*47.87/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


Al_Element_Weight_Percentages = TiAl_Input_drop["Al(at%)"]*26.98/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996+ TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


Mn_Element_Weight_Percentages = TiAl_Input_drop["Mn(at%)"]*54.94/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


Nb_Element_Weight_Percentages = TiAl_Input_drop["Nb(at%)"]*51.996/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


Cr_Element_Weight_Percentages = TiAl_Input_drop["Cr(at%)"]*52/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


Mo_Element_Weight_Percentages = TiAl_Input_drop["Mo(at%)"]*95.95/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


V_Element_Weight_Percentages = TiAl_Input_drop["V(at%)"]*54.94/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


Zr_Element_Weight_Percentages = TiAl_Input_drop["Zr(at%)"]*91.22/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


Hf_Element_Weight_Percentages = TiAl_Input_drop["Hf(at%)"]*178.49/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


Ta_Element_Weight_Percentages = TiAl_Input_drop["Ta(at%)"]*180.95/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


Si_Element_Weight_Percentages = TiAl_Input_drop["Si(at%)"]*20.09/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


W_Element_Weight_Percentages = TiAl_Input_drop["W(at%)"]*183.84/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


C_Element_Weight_Percentages = TiAl_Input_drop["C(at%)"]*12.01/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


B_Element_Weight_Percentages = TiAl_Input_drop["B(at%)"]*10.81/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


Fe_Element_Weight_Percentages = TiAl_Input_drop["Fe(at%)"]*55.85/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


Ru_Element_Weight_Percentages = TiAl_Input_drop["Ru(at%)"]*101.07/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


Re_Element_Weight_Percentages = TiAl_Input_drop["Re(at%)"]*186.21/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


Ni_Element_Weight_Percentages = TiAl_Input_drop["Ni(at%)"]*58.69/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


O_Element_Weight_Percentages = TiAl_Input_drop["O(at%)"]*16/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


Y_Element_Weight_Percentages = TiAl_Input_drop["Y(at%)"]*88.91/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)


TiB2_Element_Weight_Percentages = TiAl_Input_drop["TiB2(at%)"]*69.49/(TiAl_Input_drop["Ti(at%)"]*47.87 
                              + TiAl_Input_drop["Al(at%)"]*26.98 + TiAl_Input_drop["Mn(at%)"]*54.94 
                              + TiAl_Input_drop["Nb(at%)"]*51.996 + TiAl_Input_drop["Cr(at%)"]*52 
                              + TiAl_Input_drop["Mo(at%)"]*95.95 + TiAl_Input_drop["V(at%)"]*54.94 
                              + TiAl_Input_drop["Zr(at%)"]*91.22 + TiAl_Input_drop["Hf(at%)"]*178.49 
                              + TiAl_Input_drop["Ta(at%)"]*180.95 + TiAl_Input_drop["Si(at%)"]*20.09 
                              + TiAl_Input_drop["W(at%)"]*183.84 + TiAl_Input_drop["C(at%)"]*12.01 
                              + TiAl_Input_drop["B(at%)"]*10.81 + TiAl_Input_drop["Fe(at%)"]*55.85 
                              + TiAl_Input_drop["Ru(at%)"]*101.07 + TiAl_Input_drop["Re(at%)"]*186.21 
                              + TiAl_Input_drop["Ni(at%)"]*58.69 + TiAl_Input_drop["O(at%)"]*16 
                              + TiAl_Input_drop["Y(at%)"]*88.91 + TiAl_Input_drop["TiB2(at%)"]*69.49)

2.8 Additional Columns for Elements Weight Percentages (wt%) Into Dataframe¶

Back To Top

After calculating the weight percentages (wt%) of each element across each column for and for each row and for each alloy, their respective values were added into newly defined respective dataframes after the TiB2(at%) dataframe.

# Insert weight percentages of every element along each newly defined column after column TiB2(at%) for every alloy in every row

TiAl_Input_drop.insert(27,"Ti(wt%)", Ti_Element_Weight_Percentages)
TiAl_Input_drop.insert(28,"Al(wt%)", Al_Element_Weight_Percentages)
TiAl_Input_drop.insert(29,"Mn(wt%)", Mn_Element_Weight_Percentages)
TiAl_Input_drop.insert(30,"Nb(wt%)", Nb_Element_Weight_Percentages)
TiAl_Input_drop.insert(31,"Cr(wt%)", Cr_Element_Weight_Percentages)
TiAl_Input_drop.insert(32,"Mo(wt%)", Mo_Element_Weight_Percentages)
TiAl_Input_drop.insert(33,"V(wt%)", V_Element_Weight_Percentages)
TiAl_Input_drop.insert(34,"Zr(wt%)", Zr_Element_Weight_Percentages)
TiAl_Input_drop.insert(35,"Hf(wt%)", Hf_Element_Weight_Percentages)
TiAl_Input_drop.insert(36,"Ta(wt%)", Ta_Element_Weight_Percentages)
TiAl_Input_drop.insert(37,"Si(wt%)", Si_Element_Weight_Percentages)
TiAl_Input_drop.insert(38,"W(wt%)", W_Element_Weight_Percentages)
TiAl_Input_drop.insert(39,"C(wt%)", C_Element_Weight_Percentages)
TiAl_Input_drop.insert(40,"B(wt%)", B_Element_Weight_Percentages)
TiAl_Input_drop.insert(41,"Fe(wt%)", Fe_Element_Weight_Percentages)
TiAl_Input_drop.insert(42,"Ru(wt%)", Ru_Element_Weight_Percentages)
TiAl_Input_drop.insert(43,"Re(wt%)", Re_Element_Weight_Percentages)
TiAl_Input_drop.insert(44,"Ni(wt%)", Ni_Element_Weight_Percentages)
TiAl_Input_drop.insert(45,"O(wt%)", O_Element_Weight_Percentages)
TiAl_Input_drop.insert(46,"Y(wt%)", Y_Element_Weight_Percentages)
TiAl_Input_drop.insert(47,"TiB2(wt%)", TiB2_Element_Weight_Percentages)


TiAl_Input_drop

2.9 Cost Calculation (USD/kg)¶

Back To Top

Once the weight percentages of each element was calculated, it was essential to calculate the cost which will serve as one of the target variables (y) that will be split into testing and training datasets by using the elements compositions weight percentages (wt%) and the individual elemental cost values which are in (USD/kg).

The cost of each element was calculated by multiplying the individual respective cost value (USD/kg) of each element by the corresponding dataframe containing all their weight percentages (wt%) across all rows for every alloy.

# Input cost calculation for each alloy in each row using the elements weight percentages (wt%) and elements costs ($)

Calculated_Cost = (TiAl_Input_drop["Ti(wt%)"]*11.70 + TiAl_Input_drop["Al(wt%)"]*1.79 + TiAl_Input_drop["Mn(wt%)"]*1.82
                + TiAl_Input_drop["Nb(wt%)"]*85.60 + TiAl_Input_drop["Cr(wt%)"]*9.40 + TiAl_Input_drop["Mo(wt%)"]*40.10 
                + TiAl_Input_drop["V(wt%)"]*385 + TiAl_Input_drop["Zr(wt%)"]*37.10 + TiAl_Input_drop["Hf(wt%)"]*900 
                + TiAl_Input_drop["Ta(wt%)"]*312 + TiAl_Input_drop["Si(wt%)"]*1.70 + TiAl_Input_drop["W(wt%)"]*35.30 
                + TiAl_Input_drop["C(wt%)"]*0.122 + TiAl_Input_drop["B(wt%)"]*3.68 + TiAl_Input_drop["Fe(wt%)"]*0.424 
                + TiAl_Input_drop["Ru(wt%)"]*10600 + TiAl_Input_drop["Re(wt%)"]*4150 + TiAl_Input_drop["Ni(wt%)"]*13.90 
                + TiAl_Input_drop["O(wt%)"]*0.154 + TiAl_Input_drop["Y(wt%)"]*31 + TiAl_Input_drop["TiB2(wt%)"]*800)

2.10 Additional Columns for Cost Calculation (USD/kg) Into Dataframe¶

Back To Top

Once the cost of each element was calculated, the respective values across all rows for every alloy were added into a newly defined Cost(USD/kg) column after the Elongation (Log 10) column and this Cost(USD/kg) column will contain the full cost dataset values which will be one of the target variables (y) and be split into the training and test datasets.

# Insert cost of every alloy along a newly defined column after the Elongation (Log 10) column for every alloy in every row

TiAl_Input_drop.insert(27,"Cost (USD/kg)", Calculated_Cost)

TiAl_Input_drop

2.11 Categorical Variables Processing (process_text Function)¶

Back To Top

This dataset slightly differs from the one used in coursework 1. Categorical variables have been placed in this dataset and it is assumed that they are essential for the predictions of the yield strength, tensile strength and Elongation (Log 10). This is because the categorical variables consist of the synthesis methods used for the production of the (γ) and ((α2) + (γ)) titanium alumimides as well as the respective microstructure and theoratically the synthesis methods and microstructure formation can have an impact on the 3 mechanical properties.

Therefore, the first step to process the 2 columns of categorical variables (Synthesis Methods and Microstructure) is to simplify the text contents in both columns to their basic forms using stemming while removing the punctuations and converting them to a list with Ngram tokenising. The process_text function performs the function of converting the text (string) to a lower case, applies stemming and Ngram tokenisation and returns the tokenised text as a list into the 2 respective columns after application.

def process_text(text, n = 1):
    
    """
    Takes in a string of text, then performs the following:
    1. Convert text to lower case and remove all punctuation
    2. Optionally apply stemming
    3. Apply Ngram Tokenisation
    4. Returns the tokenised text as a list
    """
    # Define the SnowballStemmer module
    Stemmer = SnowballStemmer(language = "english")
    
    # Define the input text variable for the process_text function as a string
    text = str(text)
    
    # Convert all text to lower case for the input text string variable
    text = text.lower()
    
    # Remove all punctuation for the input text string variable
    text = re.sub(r"[^a-zA-Z0-9\s]", "", text)

    # Create the token words for the input text string variable and converting them into a list
    tokens = [token for token in text.split(" ") if token != ""]
    
    # Create an empty array to append the finalized token list words after stemming using the SnowballStemmer module 
    tokens_final = []
    
    # Stem the token list words from the input text string variable and append the results into the tokens_final empty array
    for word in tokens:
        Stemmed = Stemmer.stem(word)
        tokens_final.append(Stemmed)
        
    # Perform ngrams tokenisation of the finalized stemmed token list words from the input text string variable and return them
    tokenised = list(ngrams(tokens_final, n = n))
    
    return tokenised

2.12 Categorical Variables Processing (remove_punctuations Function)¶

Back To Top

After the stemmed text has been returned as a tokenised list, the output list of tokenised words will contain punctuations which need to be removed as categorical encoding will follow after the tokenised text list has been returned into the 2 categorical columns (Synthesis Methods and Microstructure). Therefore, the remove_punctuations function will take in the tokenised text as a string and have the punctuations removed accordingly after application.

def remove_punctuations(text):
    
    """
    Takes in a string of text, then performs the following:
    1. Removes all punctuation
    
    """
    
    for punctuation in string.punctuation:
        text = text.replace(punctuation, "")
    return text

2.13 Categorical Variables Processing (Tokenising and Stemming)¶

Back To Top

Once the process_text function has been made, it is applied throughout the Synthesis Methods column (column 22) and Microstructure column (column 23). The text has all been converted to a lower case as a string with punctuations removal with Ngram tokenisation to create the list of tokenised words with stemming to simplify the tokenised text words and return the tokenised text words as a list back to the 2 respective categorical variables columns.

# Apply the process_text function to the Synthesis Methods column

TiAl_Input_drop["Synthesis Methods"] = TiAl_Input_drop["Synthesis Methods"].apply(process_text)

TiAl_Input_drop

# Apply the process_text function to the Microstructure column

TiAl_Input_drop["Microstructure"] = TiAl_Input_drop["Microstructure"].apply(process_text)

TiAl_Input_drop

2.14 Categorical Variables Processing (Punctuations Removal After Tokenising and Stemming)¶

Back To Top

Once the simplified tokenised text words have been returned to the respective categorical columns (Synthesis Methods and Microstructure) as a list, the next step would be to convert the text words list in both columns back to a string as there are some punctuations in the lists and they cannot be removed while the words are in a list format. Therefore, the "".join(map(str, )) method was used to map the text list into a string for all text words in the list for both columns.

Once that has been done, it is now possible to remove the punctuations by applying the remove_punctuations function defined earlier to remove the punctuations in all the string text words in both columns. The next 4 cells below illustrates how the text list for both columns were converted into strings with punctuations removal.

# Convert the returned list of tokenised words back to a string format for the synthesis methods column

TiAl_Input_drop["Synthesis Methods"] = ["".join(map(str, a)) for a in TiAl_Input_drop["Synthesis Methods"]]

TiAl_Input_drop

# Remove the punctuations from the string formatted tokenised words for the synthesis methods column

TiAl_Input_drop["Synthesis Methods"] = TiAl_Input_drop["Synthesis Methods"].apply(remove_punctuations)

TiAl_Input_drop

# Convert the returned list of tokenised words back to a string format for the microstructure column

TiAl_Input_drop["Microstructure"] = ["".join(map(str, a)) for a in TiAl_Input_drop["Microstructure"]]

TiAl_Input_drop

# Remove the punctuations from the string formatted tokenised words for the microstructure column

TiAl_Input_drop["Microstructure"] = TiAl_Input_drop["Microstructure"].apply(remove_punctuations)

TiAl_Input_drop

2.15 Categorical Variables Processing (Label Encoding)¶

Back To Top

After the processing of the 2 categorical variable columns (Synthesis Methods and Microstructure), it is essential that they have be encoded into a numerical vector variable in order for any machine learning learning methods to process the information contained in both columns. Therefore, among the various encoding methods, label encoding was chosen as it has the best capability to retain each of the 2 columns while assinging a numerical vector for each type of synthesis method present within the Synthesis Methods column and for each type of microstructure present within the Microstructure column.

The retaining of the 2 columns is essential as due to the retaining of the original number of columns as 2 with label encoding for both categorical variables, it will be easier to work with the pearson correlations of the synthesis methods and the microstructure with the yield strength, tensile strength and Elongation (Log 10).

This will be be different with one hot encoding as both categorical columns will be split according to the number of different variables contained within both columns with zeros and ones. This will then increase the number of columns for the categorical variables from 2 and match with the total number of different variables contained within both columns. This will also make it difficult to work with the pearson correlations as there will be too many columns to deal with for the categorical variables and the relationships between the original 2 categorical variable columns and the mechanical properties (yield strength, tensile strength and Elongation (Log 10) cannot be obtained anymore which is why label encoding was used instead of one hot encoding.

# Apply label encoding to the synthesis methods column

Label_Encoder = LabelEncoder()

TiAl_Input_drop["Synthesis Methods"] = Label_Encoder.fit_transform(TiAl_Input_drop["Synthesis Methods"])

TiAl_Input_drop

# Apply label encoding to the microstructure column

TiAl_Input_drop["Microstructure"] = Label_Encoder.fit_transform(TiAl_Input_drop["Microstructure"])

TiAl_Input_drop

2.16 Drop Elements Weight Percentages (wt%) Columns¶

Back To Top

Once the label encoding of the 2 categorical variables columns (Synthesis Methods and Microstructure) has been done, the weight percentages of the elements (wt%) no longer have a purpose as their purpose was to be used in the calculation of the Cost (USD/kg) for each variation of the (γ) and ((α2) + (γ)) titanium alumimides persent in the dataset. Therefore, the the weight percentages of the elements (wt%) were removed accordingly.

# Remove the elements weight percentages (wt%)

Encoded_Dataset = TiAl_Input_drop.drop(["Ti(wt%)", "Al(wt%)","Mn(wt%)", "Nb(wt%)", "Cr(wt%)", "Mo(wt%)", "V(wt%)","Zr(wt%)", 
                                        "Hf(wt%)", "Ta(wt%)", "Si(wt%)", "W(wt%)", "C(wt%)","B(wt%)", "Fe(wt%)", "Ru(wt%)", 
                                        "Re(wt%)", "Ni(wt%)", "O(wt%)","Ti(wt%)", "Y(wt%)", "TiB2(wt%)"], axis=1)

Encoded_Dataset

2.17 Pearson Correlations¶

Back To Top

The Pearson correlations between calculated cost values, yield strength, tensile strength and Elongation (Log 10) and each element's atomic percentages (%) with the encoded categorical variables (Synthesis Methods and Microstructure) was essential as it can estimate how well the (target variables (y) (cost, yield strength, tensile strength and Elongation (Log 10)) across all rows for every alloy, correlate with the independent variable (X) (atomic percentages (at%) of each element present in every alloy together with the encoded categorical variables (Synthesis Methods and Microstructure)) along each column.

The larger the positive Pearson correlation, the greater the influence of that particular independent variable (X) will be in predicting the output costs, yield strength, tensile strength and Elongation (Log 10). The larger the negative Pearson correlation, it is an indication that that particular independent variable (X) and the respective dependent target variable (y) are negatively correlated. The Pearson correlations were calculated using the pearsonr function from the scipy.state module.

# Determine Pearson Correlations to check how each parameter in each column correlates with the calculated input cost

Pearson_Ti_Cost = pearsonr(Encoded_Dataset["Ti(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Al_Cost = pearsonr(Encoded_Dataset["Al(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Mn_Cost = pearsonr(Encoded_Dataset["Mn(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Nb_Cost = pearsonr(Encoded_Dataset["Nb(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Cr_Cost = pearsonr(Encoded_Dataset["Cr(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0] 
Pearson_Mo_Cost = pearsonr(Encoded_Dataset["Mo(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_V_Cost = pearsonr(Encoded_Dataset["V(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Zr_Cost = pearsonr(Encoded_Dataset["Zr(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Hf_Cost = pearsonr(Encoded_Dataset["Hf(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Ta_Cost = pearsonr(Encoded_Dataset["Ta(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Si_Cost = pearsonr(Encoded_Dataset["Si(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_W_Cost = pearsonr(Encoded_Dataset["W(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_C_Cost = pearsonr(Encoded_Dataset["C(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_B_Cost = pearsonr(Encoded_Dataset["B(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Fe_Cost = pearsonr(Encoded_Dataset["Fe(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Ru_Cost = pearsonr(Encoded_Dataset["Ru(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Re_Cost = pearsonr(Encoded_Dataset["Re(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Ni_Cost = pearsonr(Encoded_Dataset["Ni(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_O_Cost = pearsonr(Encoded_Dataset["O(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Y_Cost = pearsonr(Encoded_Dataset["Y(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_TiB2_Cost = pearsonr(Encoded_Dataset["TiB2(at%)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Test_Temperature_Cost = pearsonr(Encoded_Dataset["Test Temperature (°C)"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Synthesis_Cost = pearsonr(Encoded_Dataset["Synthesis Methods"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Microstructure_Cost = pearsonr(Encoded_Dataset["Microstructure"], Encoded_Dataset["Cost (USD/kg)"])[0]
Pearson_Cost_Cost = pearsonr(Encoded_Dataset["Cost (USD/kg)"], Encoded_Dataset["Cost (USD/kg)"])[0]

# Determine Pearson Correlations to check how each parameter in each column correlates with the yield strength

Pearson_Ti_Yield_Strength = pearsonr(Encoded_Dataset["Ti(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Al_Yield_Strength = pearsonr(Encoded_Dataset["Al(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Mn_Yield_Strength = pearsonr(Encoded_Dataset["Mn(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Nb_Yield_Strength = pearsonr(Encoded_Dataset["Nb(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Cr_Yield_Strength = pearsonr(Encoded_Dataset["Cr(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0] 
Pearson_Mo_Yield_Strength = pearsonr(Encoded_Dataset["Mo(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_V_Yield_Strength = pearsonr(Encoded_Dataset["V(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Zr_Yield_Strength = pearsonr(Encoded_Dataset["Zr(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Hf_Yield_Strength = pearsonr(Encoded_Dataset["Hf(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Ta_Yield_Strength = pearsonr(Encoded_Dataset["Ta(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Si_Yield_Strength = pearsonr(Encoded_Dataset["Si(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_W_Yield_Strength = pearsonr(Encoded_Dataset["W(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_C_Yield_Strength = pearsonr(Encoded_Dataset["C(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_B_Yield_Strength = pearsonr(Encoded_Dataset["B(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Fe_Yield_Strength = pearsonr(Encoded_Dataset["Fe(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Ru_Yield_Strength = pearsonr(Encoded_Dataset["Ru(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Re_Yield_Strength = pearsonr(Encoded_Dataset["Re(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Ni_Yield_Strength = pearsonr(Encoded_Dataset["Ni(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_O_Yield_Strength = pearsonr(Encoded_Dataset["O(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Y_Yield_Strength = pearsonr(Encoded_Dataset["Y(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_TiB2_Yield_Strength = pearsonr(Encoded_Dataset["TiB2(at%)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Test_Temperature_Yield_Strength = pearsonr(Encoded_Dataset["Test Temperature (°C)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Synthesis_Yield_Strength = pearsonr(Encoded_Dataset["Synthesis Methods"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Microstructure_Yield_Strength = pearsonr(Encoded_Dataset["Microstructure"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Cost_Yield_Strength = pearsonr(Encoded_Dataset["Cost (USD/kg)"], Encoded_Dataset["Yield Strength (MPa)"])[0]
Pearson_Yield_Yield_Strength = pearsonr(Encoded_Dataset["Yield Strength (MPa)"], Encoded_Dataset["Yield Strength (MPa)"])[0]

# Determine Pearson Correlations to check how each parameter in each column correlates with the tensile strength

Pearson_Ti_Tensile_Strength = pearsonr(Encoded_Dataset["Ti(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Al_Tensile_Strength = pearsonr(Encoded_Dataset["Al(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Mn_Tensile_Strength = pearsonr(Encoded_Dataset["Mn(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Nb_Tensile_Strength = pearsonr(Encoded_Dataset["Nb(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Cr_Tensile_Strength = pearsonr(Encoded_Dataset["Cr(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0] 
Pearson_Mo_Tensile_Strength = pearsonr(Encoded_Dataset["Mo(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_V_Tensile_Strength = pearsonr(Encoded_Dataset["V(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Zr_Tensile_Strength = pearsonr(Encoded_Dataset["Zr(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Hf_Tensile_Strength = pearsonr(Encoded_Dataset["Hf(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Ta_Tensile_Strength = pearsonr(Encoded_Dataset["Ta(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Si_Tensile_Strength = pearsonr(Encoded_Dataset["Si(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_W_Tensile_Strength = pearsonr(Encoded_Dataset["W(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_C_Tensile_Strength = pearsonr(Encoded_Dataset["C(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_B_Tensile_Strength = pearsonr(Encoded_Dataset["B(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Fe_Tensile_Strength = pearsonr(Encoded_Dataset["Fe(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Ru_Tensile_Strength = pearsonr(Encoded_Dataset["Ru(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Re_Tensile_Strength = pearsonr(Encoded_Dataset["Re(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Ni_Tensile_Strength = pearsonr(Encoded_Dataset["Ni(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_O_Tensile_Strength = pearsonr(Encoded_Dataset["O(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Y_Tensile_Strength = pearsonr(Encoded_Dataset["Y(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_TiB2_Tensile_Strength = pearsonr(Encoded_Dataset["TiB2(at%)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Test_Temperature_Tensile_Strength = pearsonr(Encoded_Dataset["Test Temperature (°C)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Synthesis_Tensile_Strength = pearsonr(Encoded_Dataset["Synthesis Methods"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Microstructure_Tensile_Strength = pearsonr(Encoded_Dataset["Microstructure"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Cost_Tensile_Strength = pearsonr(Encoded_Dataset["Cost (USD/kg)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Yield_Tensile_Strength = pearsonr(Encoded_Dataset["Yield Strength (MPa)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]
Pearson_Tensile_Tensile_Strength = pearsonr(Encoded_Dataset["Tensile Strength (MPa)"], Encoded_Dataset["Tensile Strength (MPa)"])[0]

# Determine Pearson Correlations to check how each parameter in each column correlates with the elongation

Pearson_Ti_Elongation = pearsonr(TiAl_Input_drop["Ti(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Al_Elongation = pearsonr(TiAl_Input_drop["Al(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Mn_Elongation = pearsonr(TiAl_Input_drop["Mn(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Nb_Elongation = pearsonr(TiAl_Input_drop["Nb(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Cr_Elongation = pearsonr(TiAl_Input_drop["Cr(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0] 
Pearson_Mo_Elongation = pearsonr(TiAl_Input_drop["Mo(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_V_Elongation = pearsonr(TiAl_Input_drop["V(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Zr_Elongation = pearsonr(TiAl_Input_drop["Zr(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Hf_Elongation = pearsonr(TiAl_Input_drop["Hf(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Ta_Elongation = pearsonr(TiAl_Input_drop["Ta(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Si_Elongation = pearsonr(TiAl_Input_drop["Si(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_W_Elongation = pearsonr(TiAl_Input_drop["W(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_C_Elongation = pearsonr(TiAl_Input_drop["C(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_B_Elongation = pearsonr(TiAl_Input_drop["B(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Fe_Elongation = pearsonr(TiAl_Input_drop["Fe(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Ru_Elongation = pearsonr(TiAl_Input_drop["Ru(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Re_Elongation = pearsonr(TiAl_Input_drop["Re(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Ni_Elongation = pearsonr(TiAl_Input_drop["Ni(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_O_Elongation = pearsonr(TiAl_Input_drop["O(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Y_Elongation = pearsonr(TiAl_Input_drop["Y(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_TiB2_Elongation = pearsonr(TiAl_Input_drop["TiB2(at%)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Test_Temperature_Elongation = pearsonr(TiAl_Input_drop["Test Temperature (°C)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Synthesis_Elongation = pearsonr(Encoded_Dataset["Synthesis Methods"], Encoded_Dataset["Log 10 (Elongation)"])[0]
Pearson_Microstructure_Elongation = pearsonr(Encoded_Dataset["Microstructure"], Encoded_Dataset["Log 10 (Elongation)"])[0]
Pearson_Cost_Elongation = pearsonr(TiAl_Input_drop["Cost (USD/kg)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Yield_Elongation = pearsonr(TiAl_Input_drop["Yield Strength (MPa)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Tensile_Elongation = pearsonr(TiAl_Input_drop["Tensile Strength (MPa)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]
Pearson_Elongation_Elongation = pearsonr(TiAl_Input_drop["Log 10 (Elongation)"], TiAl_Input_drop["Log 10 (Elongation)"])[0]

2.18 Additional Columns for Pearson Correlations Into Independent Dataframe¶

Back To Top

Once the Pearson correlations were calculated, the respective values were added into a newly defined independent dataframe where the column name represented the 4 target variables (y) and the rows contained the names of each element present. Each intersection of every row containing the names of every element present and every column containing the 4 target variables (y), will contain the Pearson correlation between that respective element's atomic percentages (at%) and each of the 4 target variables (y).

# Insert all calculated Pearson Correlations for Cost, Yield Strength, Tensile Strength and Elongation

Pearson_Correlations = {"Yield Strength (MPa)":[Pearson_Ti_Yield_Strength, Pearson_Al_Yield_Strength, 
                                                Pearson_Mn_Yield_Strength, Pearson_Nb_Yield_Strength, 
                                                Pearson_Cr_Yield_Strength, Pearson_Mo_Yield_Strength, 
                                                Pearson_V_Yield_Strength, Pearson_Zr_Yield_Strength, 
                                                Pearson_Hf_Yield_Strength, Pearson_Ta_Yield_Strength, 
                                                Pearson_Si_Yield_Strength, Pearson_W_Yield_Strength, 
                                                Pearson_C_Yield_Strength, Pearson_B_Yield_Strength, 
                                                Pearson_Fe_Yield_Strength, Pearson_Ru_Yield_Strength, 
                                                Pearson_Re_Yield_Strength, Pearson_Ni_Yield_Strength, 
                                                Pearson_O_Yield_Strength, Pearson_Y_Yield_Strength, 
                                                Pearson_TiB2_Yield_Strength, Pearson_Test_Temperature_Yield_Strength,
                                                Pearson_Synthesis_Yield_Strength, Pearson_Microstructure_Yield_Strength,
                                                Pearson_Yield_Yield_Strength, Pearson_Yield_Tensile_Strength, 
                                                Pearson_Yield_Elongation, Pearson_Cost_Yield_Strength],
                       
                        
                        "Tensile Strength (MPa)":[Pearson_Ti_Tensile_Strength, Pearson_Al_Tensile_Strength, 
                                                  Pearson_Mn_Tensile_Strength, Pearson_Nb_Tensile_Strength, 
                                                  Pearson_Cr_Tensile_Strength, Pearson_Mo_Tensile_Strength, 
                                                  Pearson_V_Tensile_Strength, Pearson_Zr_Tensile_Strength, 
                                                  Pearson_Hf_Tensile_Strength, Pearson_Ta_Tensile_Strength, 
                                                  Pearson_Si_Tensile_Strength, Pearson_W_Tensile_Strength, 
                                                  Pearson_C_Tensile_Strength,Pearson_B_Tensile_Strength, 
                                                  Pearson_Fe_Tensile_Strength, Pearson_Ru_Tensile_Strength, 
                                                  Pearson_Re_Tensile_Strength, Pearson_Ni_Tensile_Strength, 
                                                  Pearson_O_Tensile_Strength, Pearson_Y_Tensile_Strength, 
                                                  Pearson_TiB2_Tensile_Strength, Pearson_Test_Temperature_Tensile_Strength,
                                                  Pearson_Synthesis_Tensile_Strength, Pearson_Microstructure_Tensile_Strength,
                                                  Pearson_Yield_Tensile_Strength, Pearson_Tensile_Tensile_Strength, 
                                                  Pearson_Tensile_Elongation,  Pearson_Cost_Tensile_Strength],
                       
                        
                       "Log 10 (Elongation)":[Pearson_Ti_Elongation, Pearson_Al_Elongation, Pearson_Mn_Elongation, 
                                              Pearson_Nb_Elongation, Pearson_Cr_Elongation, Pearson_Mo_Elongation, 
                                              Pearson_V_Elongation, Pearson_Zr_Elongation, Pearson_Hf_Elongation, 
                                              Pearson_Ta_Elongation, Pearson_Si_Elongation, Pearson_W_Elongation, 
                                              Pearson_C_Elongation,Pearson_B_Elongation, Pearson_Fe_Elongation, 
                                              Pearson_Ru_Elongation, Pearson_Re_Elongation, Pearson_Ni_Elongation, 
                                              Pearson_O_Elongation, Pearson_Y_Elongation, Pearson_TiB2_Elongation, 
                                              Pearson_Test_Temperature_Elongation, Pearson_Synthesis_Elongation,
                                              Pearson_Microstructure_Elongation, Pearson_Yield_Elongation, 
                                              Pearson_Tensile_Elongation, Pearson_Elongation_Elongation, 
                                              Pearson_Cost_Elongation],
                       
                       
                       "Cost (USD/kg)":[Pearson_Ti_Cost, Pearson_Al_Cost, Pearson_Mn_Cost, Pearson_Nb_Cost, Pearson_Cr_Cost, 
                                         Pearson_Mo_Cost, Pearson_V_Cost, Pearson_Zr_Cost, Pearson_Hf_Cost, Pearson_Ta_Cost, 
                                         Pearson_Si_Cost, Pearson_W_Cost, Pearson_C_Cost, Pearson_B_Cost, Pearson_Fe_Cost, 
                                         Pearson_Ru_Cost, Pearson_Re_Cost, Pearson_Ni_Cost, Pearson_O_Cost, Pearson_Y_Cost, 
                                         Pearson_TiB2_Cost, Pearson_Test_Temperature_Cost, Pearson_Synthesis_Cost, 
                                         Pearson_Microstructure_Cost, Pearson_Cost_Yield_Strength, 
                                        Pearson_Cost_Tensile_Strength, Pearson_Cost_Elongation, Pearson_Cost_Cost]}


Pearson_Correlations_DataFrame = pd.DataFrame(Pearson_Correlations, index = ["Ti (at%)", "Al (at%)", "Mn (at%)", "Nb (at%)", 
                                                                             "Cr (at%)", "Mo (at%)", "V (at%)", "Zr (at%)", 
                                                                             "Hf (at%)", "Ta (at%)", "Si (at%)", "W (at%)", 
                                                                             "C (at%)", "B (at%)", "Fe (at%)", "Ru (at%)", 
                                                                             "Re (at%)", "Ni (at%)", "O (at%)", "Y (at%)", 
                                                                             "TiB2 (at%)", "Test Temperature (°C)", 
                                                                             "Synthesis Methods", "Microstructure", 
                                                                             "Yield Strength (MPa)", "Tensile Strength (MPa)", 
                                                                             "Log 10 (Elongation)", "Cost (USD/kg)"])

Pearson_Correlations_DataFrame

3. Linear Regression Machine Learning Model¶

3.1 Linear Regression (Cost)¶

Back To Top

After the pre-processing steps to isolate the independent variables (X) (elements atomic percentages (at%)) and the defining of the target variable (y) (cost values) from the individual elements' cost values and weight percentages, both types of variables are then ready to be incorporated into the linear regression model from python's scikit-learn module.

The first step was to define the independent variables (X) which are the elements atomic percentages (at%) and the target variable (y) which are the calculated cost values. The seed value of 1234555 was chosen as it can give the lowest mean R2 error and as such, it can be used as the randomizing token for shuffling and randomizing of the datasets before splitting into the train and test sets for both the independent (X) and dependent (y) variables.

A low R2 error implies that the model can produce the predicted cost values very well with respect to the test values from the independent (X) variable (elements atomic percentages (at%)) and the R2 value can be as close to 1 which is the maximum value attainable where there is a perfect match between the predicted cost values (y_pred) and the test cost values (y_test) from the train, test split.

Once the 2 variables were defined, they were reshaped into numerical arrays according to their original dataframe dimensions using the reshape function to ensure that there are no letters and strings but only the numbers from the respective dataframes present within both variables. Following that, the training and testing variables were defined for the independent (X) and dependent (y) variables to store the split rows of both variables.

X_train was used to store the training dataset after splitting which will contain 80% of the randomized and shuffled rows containing the columns of the elements atomic percentages (at%) while the X_test will contain the remaining 20% of the randomized and shuffled rows containing the columns of the elements atomic percentages (at%). y_train was used to store the training dataset after splitting which will contain 80% of the randomized and shuffled rows containing the column of the cost values while the y_test will contain the remaining 20% of the randomized and shuffled rows containing the columns of the cost values.

All aforementioned train and test variables were defined with the train_test_split function from python's scikit-learn module and the reshaped independent (X) and dependent (y) datasets were incorporated with a split set to 0.8 (80%) for both X_train and y_train to produce the 80% training datasets and another split set to 0.2 (20%) for both X_test and y_test to produce the remaining 20% testing datasets.

The random seed value of 1234555 was incorporated as well to lock the randomizing and shuffling variation to that particular seed value as it can produce the lowest mean R2 error and produce the best predictions with a R2 value that was very close to 1. A 80:20 split will define the number of rows for the train data for both X and y variables to be 227 (80%) and the number of rows for test data for both X and y variables to be 57 (20%)

After defining all train and test variables, the linear regression model was defined as reg. Once the linear regression model was defined, the model was incorporated with the 80% X_train and y_train datasets to train the model. After model training, the y_pred which was the variable used to define the predicted cost outputs with respect to the 20% X_test and the output costs were predicted accordingly. The next step would be then to compute the R2 value which would test the predicted cost values from y_pred with the remaining 20% y_test cost values to determine the prediction feasibility.

The final step would be to compute the mean squared error which would test the predicted cost values from y_pred with the remaining 20% y_test cost values to determine the scale of errors for the predicted cost values (y_pred) against the remaining 20% cost values (y_test). The larger the mean squared error, the more inaccurate the predictions of the cost values (y_pred) will be with respect to the 20% test values (y_test).

Based on the fixed randomizing and shuffling of the independent (X) and dependent (y) variables by the seed value of 1234555, the mean squared error was found to be the lowest possible with a mean squared value of 0.52 and the corresponding R2 value of 0.9989 which is very close to 1, indicating an excellent fit of the linear regression model between the 20% (y_test) cost values and the predicted (y_pred) values. This outcome will be used for the in-depth analysis during the graphical representation illustrated further, to produce the visualisation of how well the predicted cost values (y_pred) correlate with the 20% (y_test) cost values.

The aluminium atomic percentage Al(at%) column was removed before application of the linear regression model due to it being one of the 2 balance elements and Alchemite predicted the cost values based on the principle that the sum of the atomic percentages (at%) of all alloying elements must be equal to 1, thereby, leaving out one of the balance element compositions which was also the aluminium atomic percentage Al(at%) column values.

Therefore, the same condition was incorporated into the linear regression model to provide a fair comparison between the predicted outputs generated by the linear regression model and ones generated by Alchemite with the respective R2 values and mean squared errors to evaluate both machine learning methods' performances and limitations. However, if the aluminium atomic percentage Al(at%) column is to be preserved, the titanium atomic percentage Ti(at%) can be removed instead as it is the other balance element.

# Linear regression model application 


Encoded_Dataset_drop_Al = Encoded_Dataset.drop(["Al(at%)"], axis=1)


# Random seed for randomizing (1234555 was chosen as it gave the lowest mean R2 error)

mySeed_Cost = np.random.seed(1234555)


# Define the independent X variable (composition columns (at%)) and dependent target (y) variable (calculated input cost)

X, y = Encoded_Dataset_drop_Al.loc[:, "Ti(at%)":"TiB2(at%)"], Encoded_Dataset_drop_Al["Cost (USD/kg)"]


# Reshape the indpendent and dependent variables into numerical arrays with the appropriate dimensions

X_reshape = X.values.reshape(284, 20)
y_reshape = y.values.reshape(-1, 1)


# Insert the reshaped independent and dependent arrays into 80% training datasets and 20% test datasets

X_train, X_test, y_train, y_test = train_test_split(X_reshape, y_reshape, train_size = 0.8, test_size = 0.2, 
                                                    random_state = mySeed_Cost)


# Defining and application of Linear Regression Model

reg = LinearRegression()


# Training of Linear Regression model with the independent (X) (X_train) and dependent (y) training datasets (y_train)

Trained_Model = reg.fit(X_train, y_train)


# Output prediction of costs with the independent (X) test dataset (X_test)

y_pred = reg.predict(X_test) ###


# R2 determination based on the comparison of the dependent (y) test datasets (y_test) and predicted costs (y_pred)

r2_score(y_test, y_pred)


# Print Mean R2 value and Mean Squared Error

print("R² (Linear Regression) = %.4f" % r2_score(y_test, y_pred))
print("Mean Squared Error = %.2f" % mean_squared_error(y_test, y_pred))

R² (Linear Regression) = 0.9989
Mean Squared Error = 0.52

3.2 Dataframe Representation of Predicted Output Cost Values with Input Target Cost Values¶

Back To Top

Once the predicted cost values (y_pred) have been made according to (80:20) train-test split and the seed value of 1234555 for randomizing and shuffling, the predicted results were added into a independent dataframe which contained the remaining 20% (y_test) cost values and the predicted (y_pred) values in 2 seperate columns for visualisation of the predicted cost values (y_pred) against the 20% (y_test) actual cost values.

Nearly all of the predicted cost values (y_pred) have a near perfect match with the test cost values (y_test), hence the good R2 value of 0.9989 and the lowest mean squared error of 0.52 with a corresponding near perfect fit in the graphical representation illustrated below.

# Insert the predicted output cost values (y_pred) with the input target cost values calculated earlier (y_test)

y_pred_y_test_Comparison = pd.DataFrame(y_test, columns = ["Cost Values (y_test)"])
y_pred_y_test_Comparison.insert(1, "Cost Values (y_pred)", y_pred)
y_pred_y_test_Comparison

3.3 Graphical Representation of Predicted Output Cost Values with Input Target Cost Values¶

Back To Top

The graphical representation was made to show the visualisation of the predicted cost values (y_pred) against the 20% test cost values (y_test). The blue scatter plot represents the correlation relationship between the predicted cost values (y_pred) against the 20% test cost values (y_test).

The trendline was also calculated using a polynomial fit to produce the trendline of the blue scatter plot with its equation displayed. The R2 values produced from the linear regression model and the R2 value calculated from the trendline were also displayed accordingly. The R2 value from the linear regression model was 0.9989 while the one calculated from the trendline was 0.999.

Based on the comparison of the R2 values from both the linear regression model and the one from the trendline, both are almost identical and are very close to 1 due to the blue scatter plot having a near perfect fit of the predicted cost values (y_pred) against the 20% test cost values (y_test) and the red trendline having a near perfect match with the locations of the datapoints of the blue scatter plot of the predicted cost values (y_pred) against the 20% test cost values (y_test).

This graphical representation also amplifies the fact that the R2 values and the mean squared errors correspond to how well the predicted output of the linear regression model fits with the test data in general and the lower the mean squared error, the greater the predictions based on the test values and the lower the errors between each predicted output value and corresponding test value along each row.

# Create a graph of predicted output cost values (y_pred) against the calculated input target cost values (y_test)

# Graph is based on seed 1234555 which gives lowest mean squared errors

Cost_Plot = y_pred_y_test_Comparison.plot.scatter(x = "Cost Values (y_test)", y = "Cost Values (y_pred)", c="Blue", 
                                                  figsize = (10, 7))


Legend = plt.legend(["Cost Values (y_pred) Vs. Cost Values (y_test)"], title = None, bbox_to_anchor = (1, 0.9), 
                    loc = "lower left")

plt.gca().add_artist(Legend)


# Display trendline, respective equation and trendline R2 value

x = y_pred_y_test_Comparison["Cost Values (y_test)"]
y = y_pred_y_test_Comparison["Cost Values (y_pred)"]
z = np.polyfit(x, y, 1)
p = np.poly1d(z)
y_hat = np.poly1d(z)(x)

Text_Cost = f"y = {z[0]:0.3f}x{z[1]:+0.3f}\nR² (Trendline) = {r2_score(y, y_hat):0.3f}"

Trendline = plt.plot(x, p(x), c = "Red")

Legend_Reference = plt.legend(["Data Trendline"], title = None, bbox_to_anchor = (1, 0.9), loc = "upper left")


plt.title("Graph of Predicted Output Cost Values (y_pred) Vs. True Input Cost Values (y_test)")
plt.xlabel("True Input Cost Values (y_test)")
plt.ylabel("Predicted Output Cost Values (y_pred)")

print("Cost (Linear Regression)")
print(" ")
print("R² = %.4f" % r2_score(y_test, y_pred))
print("Mean Squared Error = %.2f" % mean_squared_error(y_test, y_pred))
print(" ")
print(Text_Cost)
print(" ")

plt.show()

Cost (Linear Regression)
 
R² = 0.9989
Mean Squared Error = 0.52
 
y = 0.990x+0.416
R² (Trendline) = 0.999

4. Multilayer Perceptron (MLP) Machine Learning Model¶

4.1 Multilayer Perceptron (MLP) Hyperparameters Optimization¶

Back To Top

The purpose of trying to predict the cost with the multilayer perceptron (MLP) model is to verify if the multilayer perceptron (MLP) model can replicate the similar results seen when linear regression is used where the R2 = 0.9989 with the mean squared error of 0.52. The steps used for defining and sorting the independent variables (X) (elements atomic percentages (at%)) and the the target variable (y) (cost values) into the train values and test values were the same as with the linear regression model elaborated earlier.

The random seed value of 1234555 was incorporated as well to lock the randomizing and shuffling variation to that particular seed value so as to verify if the multilayer perceptron (MLP) model can produce the lowest mean R2 error that was derived from the linear regression and reproduce the best results which consisted of the R2 value of 0.9989 and the mean squared arror of 0.52. The 80:20 split was used to split the number of rows for the train data for both X and y variables to be 227 (80%) and the number of rows for test data for both X and y variables to be 57 (20%). Once the 2 variables were defined, they were reshaped into numerical arrays according to their original dataframe dimensions using the reshape function.

After defining all train and test variables, the muitilayer perceptron (MLP) model was defined as MLP. In order to train the model, the hyperparameters had to be tuned to produce the best parameters for the model to be able to train and predict the cost values from the elements atomic percentages (at%). The hyperparameters that were commonly tuned for the muitilayer perceptron (MLP) model were the hidden layer sizes, activation function, solver, alpha value, learning rate and maximum number of iterations.

The hidden layer sizes will determine how many hidden layers and how many neurons are to be used in every hidden layer. The number of hidden layers and neurons in the hidden layers are usually not easy to determine manually as the selection of the number of hidden layers and the number of neurons in the hidden layers are specific to different types of problems and the respective number of input and output variables. The best way to determine them will be through the RandomizedSearchCV and the RandomizedSearchCV will consist of a range of hidden layer neurons starting from 1 to 100 to choose from.

The activation function will determine how the summation of weights of the input feature variables in each layer of the neural network which are in this case, the elements atomic percentages (at%) is transformed into an output which will be the predicted cost values. As the multilayer perceptron (MLP) model can operate in linear or non-linear mode, the loss function which is essential in the solver function hyperparameter, must be differentiable so that the deep gradient descent can occur through differentiation with backwards propagation to determine the contributions to the loss function errors through all the layers and neurons in the layers and the method to attain the differentiable loss function is to use either a linear or non-linear function as the activation function. For cost prediction, as the linear regression was able to produce a high R2 value (R2 = 0.9989) with a mean squared error of 0.52, it can be assumed that a linear or non-linear activation function can be applied for the cost prediction and the results produced by the multilayer perceptron (MLP) model will be similar to that of the linear regression model if it chooses either the linear or non-linear activation function. However, this may not be the same for the yield strength, tensile strength and Elongation (Log 10) as the predictions and relationships can be non-linear. Therefore, the activation functions to choose from from the RandomizedSearchCV will consist of both the linear and non-linear functions.

The solver function will determine the backpropagation technique to use for the deep gradient descent with the underlying loss function. The deep gradient descent will occur through differentiation of the loss function and backwards propagation to determine the contributions to the loss function errors through all the layers and neurons in the layers. Once the contributions to the errors are discovered, it will try to adjust the weights accordingly and let the muitilayer perceptron (MLP) model run in forward propagation again with the newly adjusted weights. Deep gradient descent will then occur again to determine the sources of errors through the layers and neurons once again and the once again newly adjusted weights will be input into the muitilayer perceptron (MLP) model with forward propagation to determine more sources of errors throughout the layers and neurons. This process will keep repeating to minimise the loss function errors until the minimum area of the gradient descent curve is reached where the loss function is the smallest with the smallest amount of errors with convergence or when the loss function does not change anymore. The RandomizedSearchCV will consist of 3 solver functions to choose from.

The alpha value function is the L2 regularization penalty coefficient which is used to combat underfitting and overfitting by imposing a value for regularization. The value must be small and must be sufficient such that its not too high as it can lead to underfitting and not too low as it can lead to overfitting. The alpha value can also be integrated together with the loss function to create the regularised loss function with the application of the deep gradient descent technique used by the solver function to create the regularized deep gradient descent. The RandomizedSearchCV will consist of a range of alpha values starting from 0.0001 to 0.05 to choose from. The output will be the log of the chosen alpha value.

The learning rate will determine the number of input weights to be adjusted after each successive deep gradient descent from the differentiation of each successive loss function with successive backwards propagation which determines the successive contributions to the successive loss function errors through all the layers and neurons in the layers for reducing every successive loss function until the loss function is the smallest with the smallest amount of errors with convergence or when the loss function does not change anymore (local minima). The RandomizedSearchCV will consist of 2 types of learning rates to choose from and they are constant and adaptive.

The maximum number of iterations will determine how many forward propagations to impose through the summation of weights of the input feature variables in each layer of the neural network which are in this case, the elements atomic percentages (at%) are transformed into an output which will be the predicted cost values. The more the number of times the loss function is calculated with differentiation to allow deep gradient descent to occur with the larger the number of times the errors contributing to the loss function are determined with backwards propagation, the larger the number of times those errors discovered with each iteration will be rectified with weights adjustments to improve the overall loss function with gradient descent and forward propagation with each iteration until smallest loss function is reached (minima of the gradient descent curve) with the smallest amount of errors with convergence. The RandomizedSearchCV will consist of a range of iteration values from 1 to 10000 to choose from.

To determine the best hyperparameters, the RandomizedSearchCV function was implemented with all the aforementioned hyperparameters to tune with nested cross validation value of 5 which will consist of a 80:20 (80% train and 20% test) split for each of the already 5 times split dataset from the original dataset which already has the 80:20 (80% train and 20% test) split.

4.2 Multilayer Perceptron (MLP) (Cost)¶

Back To Top

After the pre-processing steps to isolate the independent variables (X) (elements atomic percentages (at%)) and the defining of the target variable (y) (cost values) from the individual elements' cost values and weight percentages, both types of variables were then incorporated into the muitilayer perceptron (MLP) model from python's scikit-learn module.

Once the hyperparameters to tune for the multilayer perceptron (MLP) model were defined in the Random_Grid_Cost dictionary variable, the Random_Grid_Cost together with the raw multilayer perceptron (MLP) model and a nested cross validation of 5 were defined together in the RandomizedSearchCV function, the RandomizedSearchCV function was executed and the apporpriate set of hyperparameter for the cost prediction were made. The selected hyperparameters were implemented into the raw multilayer perceptron (MLP) model and the model was trained with the 80% X_train_Cost_MLP and y_train_Cost_MLP datasets.

The y_Random_Reg_Cost_MLP_pred which was the variable used to define the predicted cost outputs with respect to the 20% X_test_Cost_MLP, was then executed with the trained multilayer perceptron (MLP) model with the selected hyperparameters according to the RandomizedSearchCV and the predicted cost values were determined. The R2 value for the prediction of the cost values with respect to the 20% y_test_Cost_MLP using the trained multilayer perceptron (MLP) with the selected hyperparameters from the RandomizedSearchCV was 0.9988 with a mean squared error of 0.52. Based on the hyperparameters chosen by the RandomizedSearchCV function, the number of hidden layers it chose was 1 with 58 neurons. This was based on the nested cross validation of the already split 80:20 (80% train and 20% test) dataset.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the solver it chose for the deep gradient descent and backwards propagation to reduce the loss function to determine the smallest loss for the loss function (gradient descent local minima) was the log-linear "lbfgs" which is also known as the limited-memory-BFGS activation function. This solver is non-linear but the logarithmic values are linear.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the alpha value it chose was 1.12 which is already a logarithmic value derived from the range of 0.0001 to 0.05. The value is not too high and not too low but this value can be problem specific and usually a value slightly larger than 1 is commonly used. Therefore, it can be a little difficult to tell if the model has been underfitted or overfitted but based on the value selected by the RandomizedSearchCV function, it can be assumed that the model is just nice due to the value of alpha not being greater than 3.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the learning rate it chose was constant which is a type of learning rate where the updating of the weights after each successive deep gradient descent is constant with improvement in every successive loss function with every successive gradient descent with back propagation until the smallest loss function is reached (minima of the gradient descent curve) with the smallest amount of errors with convergence.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the maximum number of iterations it chose was 7914. This hyperparameter indicates how many forward propagations to impose through the summation of weights of the input feature variables in each layer of the neural network which are in this case, the elements atomic percentages (at%) are transformed into the output which will be the predicted cost values. The more the number of times the loss function is calculated with differentiation to allow deep gradient descent to occur with the larger the number of times the errors contributing to the loss function are determined with backwards propagation, the larger the number of times those errors discovered with each iteration will be rectified with weights adjustments to improve the overall loss function with gradient descent and forward propagation with each iteration until the smallest loss function is reached (minima of the gradient descent curve) with the smallest amount of errors with convergence. The value of 7914 indicates that the smallest loss function is reached (minima of the gradient descent curve) with the smallest amount of errors with convergence after 7914 iterations.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the activation function it chose was "identity" which is the basic activation function that is linear. This indicates that the prediction for cost values is actually linear and since it chose the linear activation function for the cost prediction, the multilayer perceptron (MLP) model will actually work similar to that of the linear regression model where the predicted relationship with the 20% test cost values will be linear based on the seed value of 1234555 for both models. This was proven to be true where the R2 value for the multilayer perceptron (MLP) model was 0.9988 with a mean squared error of 0.52 while the linear regression model had a R2 value of 0.9989 with a mean squared error of 0.52 and the results from the linear regression had the best fit. This indicates that the best regression method for cost prediction is the linear regression technique which can be applied by both the linear regression model and the multilayer perceptron (MLP) model through the use of the "identity" linear activation function as well as with the rest of the optimized hyperparameter selected by the RandomizedSearchCV function. This outcome will be used for the in-depth analysis during the graphical representation illustrated further, to produce the visualisation of how well the predicted cost values from the multilayer perceptron (MLP) model (y_Random_Reg_Cost_MLP_pred ) correlate with the 20% (y_test_Cost_MLP) cost values.

# MLP for Cost

mySeed_MLP = np.random.seed(1234555)

# Define the MLP regressor

MLP = MLPRegressor(random_state = mySeed_MLP, shuffle = True)


# Define the independent X variable and dependent target (y) variable 

X_Cost_MLP, y_Cost_MLP = Encoded_Dataset_drop_Al.loc[:, "Ti(at%)":"TiB2(at%)"], Encoded_Dataset_drop_Al["Cost (USD/kg)"]


# Reshape the data into arrays

X_reshape_Cost_MLP = X_Cost_MLP.values.reshape(284, 20)
y_reshape_Cost_MLP = y_Cost_MLP.values.reshape(-1, 1)


# Perform the train-test split

X_train_Cost_MLP, X_test_Cost_MLP, y_train_Cost_MLP, y_test_Cost_MLP = train_test_split(X_reshape_Cost_MLP, y_reshape_Cost_MLP, 
                                                                                        train_size = 0.8, test_size = 0.2)

# Intialize the random grid 

Random_Grid_Cost = {"hidden_layer_sizes": [int(a) for a in np.arange(1,100)], 
                    "activation": ["relu", "tanh", "identity", "logistic"], 
                    "solver": ["adam", "sgd", "lbfgs"], 
                    "alpha": [float(b) for b in np.logspace(0.0001,0.05)], 
                    "learning_rate": ["constant", "adaptive"], 
                    "max_iter": np.arange(1,10000,1)}


# Apply the RandomizedSearchCV

Random_Reg_Cost_MLP = RandomizedSearchCV(MLP, Random_Grid_Cost, n_jobs =- 1, cv = 5)


# Fit MLP using the optimized hyperparameters

Random_Reg_Cost_MLP.fit(X_train_Cost_MLP, y_train_Cost_MLP.ravel())

Random_Reg_Cost_MLP

RandomizedSearchCV(cv=5, estimator=MLPRegressor(), n_jobs=-1,
                   param_distributions={'activation': ['relu', 'tanh',
                                                       'identity', 'logistic'],
                                        'alpha': [1.0002302850208247,
                                                  1.0025784544921414,
                                                  1.0049321365938482,
                                                  1.0072913442675548,
                                                  1.0096560904852534,
                                                  1.01202638824939,
                                                  1.014402250592935,
                                                  1.0167836905794556,
                                                  1.019170721303187,
                                                  1.0215633558891057,
                                                  1.023961607493,
                                                  1.0263654...
                                                  1.0556560693917905,
                                                  1.0581343580333953,
                                                  1.0606184647769075,
                                                  1.0631084032810711,
                                                  1.065604187236696,
                                                  1.0681058303667321,
                                                  1.0706133464263468, ...],
                                        'hidden_layer_sizes': [1, 2, 3, 4, 5, 6,
                                                               7, 8, 9, 10, 11,
                                                               12, 13, 14, 15,
                                                               16, 17, 18, 19,
                                                               20, 21, 22, 23,
                                                               24, 25, 26, 27,
                                                               28, 29, 30, ...],
                                        'learning_rate': ['constant',
                                                          'adaptive'],
                                        'max_iter': array([   1,    2,    3, ..., 9997, 9998, 9999]),
                                        'solver': ['adam', 'sgd', 'lbfgs']})

# MLP best parameters (cost)

Random_Reg_Cost_MLP.best_params_

{'solver': 'lbfgs',
 'max_iter': 7914,
 'learning_rate': 'constant',
 'hidden_layer_sizes': 58,
 'alpha': 1.1193905407767515,
 'activation': 'identity'}

# MLP best estimator (cost)

Random_Reg_Cost_MLP.best_estimator_

MLPRegressor(activation='identity', alpha=1.1193905407767515,
             hidden_layer_sizes=58, max_iter=7914, solver='lbfgs')

# MLP cost prediction

y_Random_Reg_Cost_MLP_pred = Random_Reg_Cost_MLP.predict(X_test_Cost_MLP)

print("R² Value: %.4f" % r2_score(y_test_Cost_MLP, y_Random_Reg_Cost_MLP_pred))
print("Mean Squared Error: %.2f" % mean_squared_error(y_test_Cost_MLP, y_Random_Reg_Cost_MLP_pred))

R² Value: 0.9988
Mean Squared Error: 0.52

4.3 Dataframe Representation of Predicted Output Cost Values (MLP) with Input Target Cost Values (MLP)¶

Back To Top

Once the predicted cost values from the multilayer perceptron (MLP) model (y_Random_Reg_Cost_MLP_pred) have been made according to (80:20) train-test split and the seed value of 1234555 for randomizing and shuffling with hyperparameters optimization from the RandomSearchCV function, the predicted results were added into a independent dataframe which contained the remaining 20% (y_test_Cost_MLP) cost values and the predicted (y_Random_Reg_Cost_MLP_pred) cost values in 2 seperate columns for visualisation of the predicted cost values (y_Random_Reg_Cost_MLP_pred) against the 20% (y_test_Cost_MLP) actual cost values.

Nearly all of the predicted cost values (y_Random_Reg_Cost_MLP_pred) have a near perfect match with the test cost values (y_test_Cost_MLP), hence the good R2 value of 0.9988 and the lowest mean squared error of 0.52 with a corresponding near perfect fit in the graphical representation illustrated below. The R2 value also has a near perfect match with the one determined by the linear regression model which was 0.9989 and the mean squared error was exactly the same (0.52).

# Insert the MLP predicted output cost values with the input target cost values

MLP_Cost_Comparison = pd.DataFrame(y_test_Cost_MLP, columns = ["Cost Values MLP (y_test)"])
MLP_Cost_Comparison.insert(1, "Cost Values MLP (y_pred)", y_Random_Reg_Cost_MLP_pred)
MLP_Cost_Comparison

4.4 Graphical Representation of Predicted Output Cost Values (MLP) with Input Target Cost Values (MLP)¶

Back To Top

The graphical representation was made to show the visualisation of the predicted cost values from the hyperparameter optimized multilayer perceptron (MLP) model (y_Random_Reg_Cost_MLP_pred) against the 20% test cost values (y_test_Cost_MLP). The blue scatter plot represents the correlation relationship between the predicted cost values (y_Random_Reg_Cost_MLP_pred) against the 20% test cost values (y_test_Cost_MLP).

The trendline was also calculated using a polynomial fit to produce the trendline of the blue scatter plot with its equation displayed. The R2 values produced from the multilayer perceptron (MLP) model and the R2 value calculated from the trendline were also displayed accordingly. The R2 value from the multilayer perceptron (MLP) model was 0.9988 while the one calculated from the trendline was 0.990.

Based on the comparison of the R2 values from both the multilayer perceptron (MLP) model and the one from the trendline, both are almost identical and are very close to 1 due to the blue scatter plot having a near perfect fit of the predicted cost values (y_Random_Reg_Cost_MLP_pred) against the 20% test cost values (y_test_Cost_MLP) and the red trendline having a near perfect match with the locations of the datapoints of the blue scatter plot of the predicted cost values (y_Random_Reg_Cost_MLP_pred) against the 20% test cost values (y_test_Cost_MLP).

This graphical representation also amplifies the fact that the R2 values and the mean squared errors correspond to how well the predicted output of the multilayer perceptron (MLP) model fits with the test data in general and the lower the mean squared error, the greater the predictions based on the test values and the lower the errors between each predicted output value and corresponding test value along each row.

# MLP Cost Graph

Cost_Plot_MLP = MLP_Cost_Comparison.plot.scatter(x = "Cost Values MLP (y_test)", y = "Cost Values MLP (y_pred)", c="Blue", 
                                                 figsize = (10, 7))


Legend_Cost_MLP = plt.legend(["Cost Values MLP (y_pred) Vs. Cost Values MLP (y_test)"], title = None, bbox_to_anchor = (1, 0.9), 
                             loc = "lower left")

plt.gca().add_artist(Legend_Cost_MLP)


# Display trendline, respective equation and trendline R2 value

x_Cost_MLP = MLP_Cost_Comparison["Cost Values MLP (y_test)"]
y_Cost_MLP = MLP_Cost_Comparison["Cost Values MLP (y_pred)"]
z_Cost_MLP = np.polyfit(x_Cost_MLP, y_Cost_MLP, 1)
p_Cost_MLP = np.poly1d(z_Cost_MLP)
y_hat_Cost_MLP = np.poly1d(z_Cost_MLP)(x_Cost_MLP)

Text_Cost_MLP = f"y = {z_Cost_MLP[0]:0.3f}x{z_Cost_MLP[1]:+0.3f}\nR² (Trendline) = {r2_score(y_Cost_MLP, y_hat_Cost_MLP):0.3f}"

Trendline_MLP = plt.plot(x_Cost_MLP, p(x_Cost_MLP), c = "Red")

Legend_Reference_MLP = plt.legend(["Data Trendline"], title = None, bbox_to_anchor = (1, 0.9), loc = "upper left")


# Plotting of MLP Cost Graph

plt.title("Graph of Predicted Output Cost Values MLP (y_pred) Vs. True Input Cost Values MLP (y_test)")
plt.xlabel("True Input Cost Values MLP (y_test)")
plt.ylabel("Predicted Output Cost Values MLP (y_pred)")

print("Cost (MLP Regression)")
print(" ")
print("R² = %.4f" % r2_score(y_test_Cost_MLP, y_Random_Reg_Cost_MLP_pred))
print("Mean Squared Error = %.2f" % mean_squared_error(y_test_Cost_MLP, y_Random_Reg_Cost_MLP_pred))
print(" ")
print(Text_Cost_MLP)
print(" ")

plt.show()

Cost (MLP Regression)
 
R² = 0.9988
Mean Squared Error = 0.52
 
y = 0.990x+0.422
R² (Trendline) = 0.999

4.5 Yield Strength Best Parameters Selection (MLP)¶

Back To Top

After using the multilayer perceptron (MLP) model to predict the cost values and comparing them with the results of the linear regression model, the next major task is to try to predict one of the three essential mechanical properties for which in this case, is the yield strength. Compared with cost, the yield strength is not easy to predict as the predicted relationship may not be linear as with cost. The values present in the yield strength column are also large which is an issue for machine learning to learn and comprehend which leads to a complex dataset with complex predictions.

The last obstacle is that if all the elements atomic percentages (at%) columns together with the encoded synthesis methods and microstructure columns are set as the independent (X) variables and the yield strength column is set as the dependent (y) variable which is to be predicted, the predictions can have a high chance to be inaccurate as some of the columns which have been set as the input columns have little to no correlations with the yield strength and their redundant correlations will cause the multilayer perceptron (MLP) model to take them into consideration and produce bad predictions with bad R2 values together with the nature of the complex dataset. The redundant correlations can be seen and compared in the pearson correlations table in section 2.17.

The first step in predicting the yield strength would be to do generalization. Generalization in this case, would be to implement the ordinary least squares (OLS) module to display all the columns that are to be used as the indpendent (X) input variables and set the dependent (y) variable to be the yield strength column. Generalization will be able to measure the p-values which are the probability values. A p-value of less than 0.05 is statistically significant and it implies that it has significance to the dependent (y) variable which is the yield strength and it rejects the null hypothesis. Any input variable with p-values larger than 0.05 will have little to no significance in the prediction of the yield strength as it retains the null hypothesis.

Therefore, the ordinary least squares (OLS) module will be executed as many times as possible with the removel of the input independent (X) variable that produces the largest p-value each time until all the leftover input independent (X) variables have their respective p-values below the threshold value of 0.05. The elimination of all input independent (X) variables using the ordinary least squares (OLS) module, were executed below until the leftover input independent (X) variables have their respective p-values below the threshold value of 0.05.

# Rename Encoded_Dataset_drop_Al to Encoded_Dataset_1

Encoded_Dataset_1 = Encoded_Dataset_drop_Al

Encoded_Dataset_1

# Execute Ordinary Least Squares Module (OLS)

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "Si(at%)", "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Re(at%)", "Ni(at%)", "O(at%)", 
                          "Y(at%)", "TiB2(at%)", "Test Temperature (°C)", "Synthesis Methods", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove C

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "Si(at%)", "W(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Re(at%)", "Ni(at%)", "O(at%)", 
                          "Y(at%)", "TiB2(at%)", "Test Temperature (°C)", "Synthesis Methods", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Re

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "Si(at%)", "W(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Ni(at%)", "O(at%)", 
                          "Y(at%)", "TiB2(at%)", "Test Temperature (°C)", "Synthesis Methods", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Y

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "Si(at%)", "W(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Ni(at%)", "O(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Synthesis Methods", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Synthesis Methods

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "Si(at%)", "W(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Ni(at%)", "O(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Ni

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "Si(at%)", "W(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "O(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Si

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "W(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "O(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Mn

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "W(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "O(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Ta

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)",
                          "W(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "O(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Hf

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)",
                          "W(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "O(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove O

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)",
                          "W(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Microstructure

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)",
                          "W(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove W

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)",
                          "B(at%)", "Fe(at%)", "Ru(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove V

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "Zr(at%)",
                          "B(at%)", "Fe(at%)", "Ru(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Zr

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)",
                          "B(at%)", "Fe(at%)", "Ru(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Ru

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)",
                          "B(at%)", "Fe(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove TiB2

X_opt = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)",
                          "B(at%)", "Fe(at%)", 
                          "Test Temperature (°C)"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_1["Yield Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

4.6 Multilayer Perceptron (MLP) (Yield Strength)¶

Back To Top

Once the ordinary least squares module was implemented and the leftover input independent (X) variables with the respective p-values of less than the threshold value of 0.05 were determined, the leftover independent (X) variables were set as the input columns and the dependent (y) variable (yield strength) was set as the output for prediction. The input independent (X) variable columns which are statistically significant in the prediction of the yield strength are the titanium (Ti(at%)), niobium (Nb(at%)), chromium (Cr(at%)), molybdenum (Mo(at%)), boron (B(at%)), iron (Fe(at%)), and the test temperature (Test Temperature (°C)) columns.

The standard configuration for the multilayer perceptron (MLP) model was similar as with the one used for the cost prediction except this time there were 2 additional tasks which were done. The first additional task was data scaling where the respective leftover independent (X) variables and the dependent (y) variable (yield strength) had to be scaled with the StandardScalar module as the values were large and the multilayer perceptron (MLP) model may not be able to process the data and predict correctly. The scaling would produce normalized smaller values with centering and the multilayer perceptron (MLP) model will be able to process them properly and predict correctly.

Just as with the multilayer perceptron (MLP) model configuration seen for the cost prediction, the RandomizedSearchCV module was used to determine the best hyperparameters which are the hidden layer sizes, activation function, solver, alpha value, learning rate and maximum number of iterations.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the number of hidden layers it chose was 1 with 99 neurons. This was based on the nested cross validation of the already split 80:20 (80% train and 20% test) dataset.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the solver it chose for the deep gradient descent and backwards propagation to reduce the loss function to determine the smallest loss for the loss function (gradient descent local minima) was the log-linear "lbfgs" which is also known as the limited-memory-BFGS activation function. This solver is non-linear but the logarithmic values are linear.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the alpha value it chose was 1.12 which is already a logarithmic value derived from the range of 0.0001 to 0.05. The value is not too high and not too low but this value can be problem specific and usually a value slightly larger than 1 is commonly used. Therefore, it can be a little difficult to tell if the model has been underfitted or overfitted but based on the value selected by the RandomizedSearchCV function, it can be assumed that the model is just nice due to the value of alpha not being greater than 3.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the learning rate it chose was constant which is a type of learning rate where the updating of the weights after each successive deep gradient descent is constant with improvement in every successive loss function with every successive gradient descent with back propagation until the smallest loss function is reached (minima of the gradient descent curve) with the smallest amount of errors with convergence.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the maximum number of iterations it chose was 9909. This hyperparameter indicates how many forward propagations to impose through the summation of weights of the input feature variables in each layer of the neural network are transformed into the output which will be the predicted yield strength. The more the number of times the loss function is calculated with differentiation to allow deep gradient descent to occur with the larger the number of times the errors contributing to the loss function are determined with backwards propagation, the larger the number of times those errors discovered with each iteration will be rectified with weights adjustments to improve the overall loss function with gradient descent and forward propagation with each iteration until the smallest loss function is reached (minima of the gradient descent curve) with the smallest amount of errors with convergence. The value of 9909 indicates that the smallest loss function is reached (minima of the gradient descent curve) with the smallest amount of errors with convergence after 9909 iterations.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the activation function it chose was "relu" which is the rectified linear activation function. This indicates that the prediction for the yield strength may be linear and since it chose the rectified linear activation function for the yield strength prediction, the multilayer perceptron (MLP) model may work in a way similar to that of the linear regression model where the predicted relationship with the 20% test yield strength values may be somewhat linear. This was true as the R2 value for the multilayer perceptron (MLP) model was 0.7218 with a normalized mean squared error of 0.28. This indicates that the multilayer perceptron (MLP) model is able to predict the yield strength values using the input variables which have the most significant impact for the prediction based on the ordinary least squares (OLS) redundant input variables elimination as well as with the rest of the optimized hyperparameter selected by the RandomizedSearchCV function. This outcome will be used for the in-depth analysis during the graphical representation illustrated further, to produce the visualisation of how well the predicted yield strength values from the multilayer perceptron (MLP) model (y_Random_Reg_Yield_MLP_pred) correlate with the 20% (y_test_Yield_Strength_MLP) yield strength values.

# MLP for Yield Strength

mySeed_MLP = np.random.seed(20)


# Define the MLP regressor

MLP = MLPRegressor(random_state = mySeed_MLP, shuffle = True)


# Define the independent (X) and dependent (y) target variable

X_Yield_Strength_MLP, y_Yield_Strength_MLP = Encoded_Dataset_1.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "B(at%)", "Fe(at%)", "Test Temperature (°C)"]], Encoded_Dataset_1["Yield Strength (MPa)"]


# Reshape the dependent (y) target variable into an array

y_reshape_Yield_Strength_MLP = y_Yield_Strength_MLP.values.reshape(-1, 1)


# Perform the train-test split

X_train_Yield_Strength_MLP, X_test_Yield_Strength_MLP, y_train_Yield_Strength_MLP, y_test_Yield_Strength_MLP = train_test_split(X_Yield_Strength_MLP, y_reshape_Yield_Strength_MLP, 
                                                                                                               train_size = 0.8, test_size = 0.2)

# Perform scaling and centering of data

SS = StandardScaler()

X_Yield_Strength_scale_MLP = SS.fit_transform(X_Yield_Strength_MLP) 
y_Yield_Strength_scale_MLP = SS.fit_transform(y_reshape_Yield_Strength_MLP)

X_train_Yield_scale_MLP = SS.fit_transform(X_train_Yield_Strength_MLP)  
X_test_Yield_scale_MLP = SS.fit_transform(X_test_Yield_Strength_MLP)
y_train_Yield_scale_MLP = SS.fit_transform(y_train_Yield_Strength_MLP)  
y_test_Yield_scale_MLP = SS.fit_transform(y_test_Yield_Strength_MLP)


# Initialize the random grid

Random_Grid_Yield = {"hidden_layer_sizes": [int(c) for c in np.arange(1,100)], 
                     "activation": ["relu", "tanh", "logistic"],
                     "solver": ["adam", "sgd", "lbfgs"], 
                     "alpha": [float(d) for d in np.logspace(0.0001,0.05)],
                     "learning_rate": ["constant", "adaptive"],
                     "max_iter": np.arange(1,10000,1)}


# Apply the RandomizedSearchCV

Random_Reg_Yield_MLP = RandomizedSearchCV(MLP, Random_Grid_Yield, n_jobs =- 1, cv = 5)


# Fit MLP using the optimized hyperparameters

Random_Reg_Yield_MLP.fit(X_train_Yield_scale_MLP, y_train_Yield_scale_MLP.ravel())

Random_Reg_Yield_MLP

RandomizedSearchCV(cv=5, estimator=MLPRegressor(), n_jobs=-1,
                   param_distributions={'activation': ['relu', 'tanh',
                                                       'logistic'],
                                        'alpha': [1.0002302850208247,
                                                  1.0025784544921414,
                                                  1.0049321365938482,
                                                  1.0072913442675548,
                                                  1.0096560904852534,
                                                  1.01202638824939,
                                                  1.014402250592935,
                                                  1.0167836905794556,
                                                  1.019170721303187,
                                                  1.0215633558891057,
                                                  1.023961607493,
                                                  1.026365489301543,
                                                  1....
                                                  1.0556560693917905,
                                                  1.0581343580333953,
                                                  1.0606184647769075,
                                                  1.0631084032810711,
                                                  1.065604187236696,
                                                  1.0681058303667321,
                                                  1.0706133464263468, ...],
                                        'hidden_layer_sizes': [1, 2, 3, 4, 5, 6,
                                                               7, 8, 9, 10, 11,
                                                               12, 13, 14, 15,
                                                               16, 17, 18, 19,
                                                               20, 21, 22, 23,
                                                               24, 25, 26, 27,
                                                               28, 29, 30, ...],
                                        'learning_rate': ['constant',
                                                          'adaptive'],
                                        'max_iter': array([   1,    2,    3, ..., 9997, 9998, 9999]),
                                        'solver': ['adam', 'sgd', 'lbfgs']})

# MLP Best Parameters (Yield Strength)

Random_Reg_Yield_MLP.best_params_

{'solver': 'lbfgs',
 'max_iter': 9909,
 'learning_rate': 'constant',
 'hidden_layer_sizes': 99,
 'alpha': 1.1115436720692664,
 'activation': 'relu'}

# MLP Best Estimator (Yield Strength)

Random_Reg_Yield_MLP.best_estimator_

MLPRegressor(alpha=1.1115436720692664, hidden_layer_sizes=99, max_iter=9909,
             solver='lbfgs')

# MLP yield strength prediction

y_Random_Reg_Yield_MLP_pred = Random_Reg_Yield_MLP.predict(X_test_Yield_scale_MLP)

print("R² Value: %.4f" % r2_score(SS.inverse_transform(y_test_Yield_scale_MLP), SS.inverse_transform(y_Random_Reg_Yield_MLP_pred)))
print("Mean Squared Error: %.2f" % mean_squared_error(y_test_Yield_scale_MLP, y_Random_Reg_Yield_MLP_pred))
print("Mean Squared Error (Non-Normalized) = %.2f" % mean_squared_error(SS.inverse_transform(y_test_Yield_scale_MLP), SS.inverse_transform(y_Random_Reg_Yield_MLP_pred)))

R² Value: 0.7218
Mean Squared Error: 0.28
Mean Squared Error (Non-Normalized) = 29317.10

4.7 Dataframe Representation of Predicted Output Yield Strength Values (MLP) with Input Target Yield Strength Values (MLP)¶

Back To Top

Once the predicted yield strength values from the multilayer perceptron (MLP) model (y_Random_Reg_Yield_MLP_pred) have been made according to (80:20) train-test split and the seed value of 20 for randomizing and shuffling with hyperparameters optimization from the RandomSearchCV function, the predicted results were added into a independent dataframe which contained the remaining 20% (y_test_Yield_Strength_MLP) yield strength values and the predicted (y_Random_Reg_Yield_MLP_pred) yield strength values in 2 seperate columns for visualisation of the predicted yield strength values (y_Random_Reg_Yield_MLP_pred) against the 20% (y_test_Yield_Strength_MLP) actual yield strength values.

The predicted yield strength values (y_Random_Reg_Yield_MLP_pred) when correlated with the test yield strength values, have a R2 value of 0.7218 with a normalized mean squared error of 0.28 which is generally good for yield strength prediction as the correlations between the yield strength with the input independent (X) variables defined by the ordinary least squared (OLS) module are generally good but not excellent. However, if all the elements atomic percentages (at%) and the encoded synthesis methods and microstructure are set as the input independent (X) variables, the R2 will definitely be worse than 0.7218 as it will incorporate the redundant correlations which have little to no effect on the prediction of the yield strength and thereby, affect the predictions and produce bad R2 values. However, the non-normalized error is 29317.10 which is high but it should be noted that the prediction values are large and it is natural for the non-normalized error value to be somewhat large.

# Insert the predicted yield strength values with the input target yield strength values

MLP_Yield_Strength_Comparison = pd.DataFrame(SS.inverse_transform(y_test_Yield_scale_MLP), columns = ["Yield Strength Values MLP (y_test)"])
MLP_Yield_Strength_Comparison.insert(1, "Yield Strength Values MLP (y_pred)", SS.inverse_transform(y_Random_Reg_Yield_MLP_pred))
MLP_Yield_Strength_Comparison

4.8 Graphical Representation of Predicted Output Yield Strength Values (MLP) with Input Target Yield Strength Values (MLP)¶

Back To Top

This graphical representation was made to show the visualisation of the predicted yield strength values from the hyperparameter optimized multilayer perceptron (MLP) model (y_Random_Reg_Yield_MLP_pred) against the 20% test yield strength values (y_test_Yield_Strength_MLP). The blue scatter plot represents the correlation relationship between the predicted yield strength values (y_Random_Reg_Yield_MLP_pred) against the 20% test yield strength values (y_test_Yield_Strength_MLP).

The trendline was also calculated using a polynomial fit to produce the trendline of the blue scatter plot with its equation displayed. The R2 value produced from the multilayer perceptron (MLP) model and the R2 value calculated from the trendline were also displayed accordingly. The R2 value from the multilayer perceptron (MLP) model was 0.7218 while the one calculated from the trendline was 0.726.

Based on the comparison of the R2 values from both the multilayer perceptron (MLP) model and the one from the trendline, both are almost identical. It should be noted that there are some outliers present which is expected due to the complexity of the yield strength prediction with respect to the indpendent input variables used according to the ordinary least squares (OLS) module. This is one of the challenges faced when predicting the yield strength. However, based on the R2 value of 0.7218 from the multilayer perceptron (MLP) model and the indpendent input variables used according to the ordinary least squares (OLS) module, it can be considered to be a good prediction with room for improvement.

The unique property of the rectified linear activation function ("relu") is that it works in such a way that the linearity will lie with a range of values between zero and infinity. This is essential for the prediction of the yield strength as yield strength values are always positive and can never be negative. The range of positive values can lie anywhere within the positive scale towards very high positive values and the upper bound of infinity provides an endless possibility of predicting the positive values of the yield strength with respect to the other optimized hyperparameters derived from the RandomizedSearchCV and the statistically significant input independent (X) variables. This phenomenon can be clearly seen from the graphical representation.

# MLP Yield Strength Graph


# Definition of x and y axes variables

Yield_Strength_Plot_MLP = MLP_Yield_Strength_Comparison.plot.scatter(x = "Yield Strength Values MLP (y_test)", y = "Yield Strength Values MLP (y_pred)", c="Blue", 
                                                                     figsize = (10, 7))


Legend_Yield_Strength_MLP = plt.legend(["Yield Strength Values MLP (y_pred) Vs. Yield Strength Values MLP (y_test)"], title = None, bbox_to_anchor = (1, 0.9), 
                                       loc = "lower left")

plt.gca().add_artist(Legend_Yield_Strength_MLP)


# Display trendline, respective equation and trendline R2 value

x_Yield_Strength_MLP = MLP_Yield_Strength_Comparison["Yield Strength Values MLP (y_test)"]
y_Yield_Strength_MLP = MLP_Yield_Strength_Comparison["Yield Strength Values MLP (y_pred)"]
z_Yield_Strength_MLP = np.polyfit(x_Yield_Strength_MLP, y_Yield_Strength_MLP, 1)
p_Yield_Strength_MLP = np.poly1d(z_Yield_Strength_MLP)
y_hat_Yield_Strength_MLP = np.poly1d(z_Yield_Strength_MLP)(x_Yield_Strength_MLP)

Text_Yield_Strength_MLP = f"y = {z_Yield_Strength_MLP[0]:0.3f}x{z_Yield_Strength_MLP[1]:+0.3f}\nR² (Trendline) = {r2_score(y_Yield_Strength_MLP, y_hat_Yield_Strength_MLP):0.3f}"

Trendline_Yield_Strength_MLP = plt.plot(x_Yield_Strength_MLP, p(x_Yield_Strength_MLP), c = "Red")

Legend_Reference_Yield_Strength_MLP = plt.legend(["Data Trendline"], title = None, bbox_to_anchor = (1, 0.9), loc = "upper left")


# Plotting of MLP Yield Strength Graph

plt.title("Graph of Predicted Output Yield Strength Values MLP (y_pred) Vs. True Input Yield Strength Values MLP (y_test)")
plt.xlabel("True Input Yield Strength Values MLP (y_test)")
plt.ylabel("Predicted Output Yield Strength Values MLP (y_pred)")

print("Yield Strength (MLP Regression)")
print(" ")
print("R² = %.4f" % r2_score(SS.inverse_transform(y_test_Yield_scale_MLP), SS.inverse_transform(y_Random_Reg_Yield_MLP_pred)))
print("Mean Squared Error = %.2f" % mean_squared_error(y_test_Yield_scale_MLP, y_Random_Reg_Yield_MLP_pred))
print("Mean Squared Error (Non-Normalized) = %.2f" % mean_squared_error(SS.inverse_transform(y_test_Yield_scale_MLP), SS.inverse_transform(y_Random_Reg_Yield_MLP_pred)))
print(" ")
print(Text_Yield_Strength_MLP)
print(" ")

plt.show()

Yield Strength (MLP Regression)
 
R² = 0.7218
Mean Squared Error = 0.28
Mean Squared Error (Non-Normalized) = 29317.10
 
y = 0.670x+187.379
R² (Trendline) = 0.726

4.9 Tensile Strength Best Parameters Selection (MLP)¶

Back To Top

After using the multilayer perceptron (MLP) model to predict the yield strength the next major task is to predict the tensile strength. Compared with cost, the tensile strength is not easy to predict as the predicted relationship may not be linear as with cost. The values present in the tensile strength column are also large which is an issue for machine learning to learn and comprehend which leads to a complex dataset with complex predictions. This same principle also applied for the yield strength prediction.

If all the elements atomic percentages (at%) columns together with the encoded synthesis methods and microstructure columns are set as the independent (X) variables and the tensile strength column is set as the dependent (y) variable which is to be predicted, the predictions can have a high chance to be inaccurate as some of the columns which have been set as the input columns have little to no correlations with the tensile strength and their redundant correlations will cause the multilayer perceptron (MLP) model to take them into consideration and produce bad predictions with bad R2 values together with the nature of the complex dataset. The redundant correlations can be seen and compared in the pearson correlations table in section 2.17.

The first step in predicting the tensile strength would be to do generalization just as for what was done for the yield strength prediction. Generalization, in this case, would be to implement the ordinary least squares (OLS) module to display all the columns that are to be used as the indpendent (X) input variables and set the dependent (y) variable to be the tensile strength column. Generalization will be able to measure the p-values which are the probability values. A p-value of less than 0.05 is statistically significant and it implies that it has significance to the dependent (y) variable which is the tensile strength and it rejects the null hypothesis. Any input variable with p-values larger than 0.05 will have little to no significance in the prediction of the tensile strength as it retains the null hypothesis.

Therefore, the ordinary least squares (OLS) module will be executed as many times as possible with the removel of the input independent (X) variable that produces the largest p-value each time until all the leftover input independent (X) variables have their respective p-values below the threshold value of 0.05. The elimination of all input independent (X) variables using the ordinary least squares (OLS) module, were executed below until the leftover input independent (X) variables have their respective p-values below the threshold value of 0.05.

# Rename Encoded_Dataset_1 to Encoded_Dataset_2

Encoded_Dataset_2 = Encoded_Dataset_1

Encoded_Dataset_2

# Execute Ordinary Least Squares (OLS) module

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "Si(at%)", "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Re(at%)", "Ni(at%)", "O(at%)", 
                          "Y(at%)", "TiB2(at%)", "Test Temperature (°C)", "Synthesis Methods", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Re

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "Si(at%)", "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Ni(at%)", "O(at%)", 
                          "Y(at%)", "TiB2(at%)", "Test Temperature (°C)", "Synthesis Methods", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Si

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Ni(at%)", "O(at%)", 
                          "Y(at%)", "TiB2(at%)", "Test Temperature (°C)", "Synthesis Methods", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Synthesis Methods

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Ni(at%)", "O(at%)", 
                          "Y(at%)", "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Y

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Ni(at%)", "O(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Hf

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Ta(at%)",
                          "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Ni(at%)", "O(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Ta

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)",
                          "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Ni(at%)", "O(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Mn

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)",
                          "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Ni(at%)", "O(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Ni

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)",
                          "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "O(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove V

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "Zr(at%)",
                          "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "O(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Zr

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)",
                          "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "O(at%)", 
                          "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove TiB2

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)",
                          "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "O(at%)", 
                          "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove O

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)",
                          "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", 
                          "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove W

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)",
                          "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", 
                          "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Ru

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)",
                          "C(at%)", "B(at%)", "Fe(at%)", 
                          "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Tensile Strength (MPa)"], exog = X_opt).fit()
regressor_OLS.summary()

4.10 Multilayer Perceptron (MLP) (Tensile Strength)¶

Back To Top

Once the ordinary least squares module was implemented and the leftover input independent (X) variables with the respective p-values of less than the threshold value of 0.05 were determined, the leftover independent (X) variables were set as the input columns and the dependent (y) variable (tensile strength) was set as the output for prediction. The input independent (X) variable columns which are statistically significant in the prediction of the yield strength are the titanium (Ti(at%)), niobium (Nb(at%)), chromium (Cr(at%)), molybdenum (Mo(at%)), carbon (C(at%)), boron (B(at%)), iron (Fe(at%)), test temperature (Test Temperature (°C)) and the microstructure (Microstructure) columns.

The standard configuration for the multilayer perceptron (MLP) model was similar as with the one used for the yield strength prediction with data scaling where the respective leftover independent (X) variables and the dependent (y) variable (tensile strength) had to be scaled with the StandardScalar module as the values were large and the multilayer perceptron (MLP) model may not be able to process the data and predict correctly. The scaling would produce normalized smaller values with centering and the multilayer perceptron (MLP) model will be able to process them properly and predict correctly.

Just as with the multilayer perceptron (MLP) model configuration seen for the cost prediction and yield strength prediction, the RandomizedSearchCV module was used to determine the best hyperparameters which are the hidden layer sizes, activation function, solver, alpha value, learning rate and maximum number of iterations.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the number of hidden layers it chose was 1 with 19 neurons. This was based on the nested cross validation of the already split 80:20 (80% train and 20% test) dataset.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the solver it chose for the deep gradient descent and backwards propagation to reduce the loss function to determine the smallest loss for the loss function (gradient descent local minima) was the log-linear "lbfgs" which is also known as the limited-memory-BFGS activation function. This solver is non-linear but the logarithmic values are linear.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the alpha value it chose was 1.03 which is already a logarithmic value derived from the range of 0.0001 to 0.05. The value is not too high and not too low but this value can be problem specific and usually a value slightly larger than 1 is commonly used. Therefore, it can be a little difficult to tell if the model has been underfitted or overfitted but based on the value selected by the RandomizedSearchCV function, it can be assumed that the model is just nice due to the value of alpha not being greater than 3.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the learning rate it chose was constant which is a type of learning rate where the updating of the weights after each successive deep gradient descent is constant with improvement in every successive loss function with every successive gradient descent with back propagation until the smallest loss function is reached (minima of the gradient descent curve) with the smallest amount of errors with convergence.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the maximum number of iterations it chose was 5572. This hyperparameter indicates how many forward propagations to impose through the summation of weights of the input feature variables in each layer of the neural network are transformed into the output which will be the predicted tensile strength. The more the number of times the loss function is calculated with differentiation to allow deep gradient descent to occur with the larger the number of times the errors contributing to the loss function are determined with backwards propagation, the larger the number of times those errors discovered with each iteration will be rectified with weights adjustments to improve the overall loss function with gradient descent and forward propagation with each iteration until the smallest loss function is reached (minima of the gradient descent curve) with the smallest amount of errors with convergence. The value of 5572 indicates that the smallest loss function is reached (minima of the gradient descent curve) with the smallest amount of errors with convergence after 5572 iterations.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the activation function it chose was "relu" which is the rectified linear activation function. This indicates that the prediction for the tensile strength may be linear and since it chose the rectified linear activation function for the tensile strength prediction, the multilayer perceptron (MLP) model may work in a way similar to that of the linear regression model where the predicted relationship with the 20% test tensile strength values may be somewhat linear. This was true as the R2 value for the multilayer perceptron (MLP) model was 0.8444 with a normalized mean squared error of 0.16. This indicates that the multilayer perceptron (MLP) model is able to predict the tensile strength values using the input variables which have the most significant impact for the prediction based on the ordinary least squares (OLS) redundant input variables elimination as well as with the rest of the optimized hyperparameter selected by the RandomizedSearchCV function. This outcome will be used for the in-depth analysis during the graphical representation illustrated further, to produce the visualisation of how well the predicted tensile strength values from the multilayer perceptron (MLP) model (y_Random_Reg_Tensile_MLP_pred) correlate with the 20% (y_test_Tensile_Strength_MLP) tensile strength values.

# MLP for Tensile Strength

mySeed_MLP = np.random.seed(14)


# Define the MLP regressor

MLP = MLPRegressor(random_state = mySeed_MLP, shuffle = True)


# Define the independent (X) variable and dependent (y) variable

X_Tensile_Strength_MLP, y_Tensile_Strength_MLP = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Test Temperature (°C)", "Microstructure"]], Encoded_Dataset_2["Tensile Strength (MPa)"]


# Reshape the dependent (y) variable into an array

y_reshape_Tensile_Strength_MLP = y_Tensile_Strength_MLP.values.reshape(-1, 1)


# Perform the train-test split

X_train_Tensile_Strength_MLP, X_test_Tensile_Strength_MLP, y_train_Tensile_Strength_MLP, y_test_Tensile_Strength_MLP = train_test_split(X_Tensile_Strength_MLP, y_reshape_Tensile_Strength_MLP, 
                                                                                                               train_size = 0.8, test_size = 0.2)

# Perform scaling and centering of data

SS = StandardScaler()

X_Tensile_Strength_scale_MLP = SS.fit_transform(X_Tensile_Strength_MLP) 
y_Tensile_Strength_scale_MLP = SS.fit_transform(y_reshape_Tensile_Strength_MLP)

X_train_Tensile_scale_MLP = SS.fit_transform(X_train_Tensile_Strength_MLP)  
X_test_Tensile_scale_MLP = SS.fit_transform(X_test_Tensile_Strength_MLP)
y_train_Tensile_scale_MLP = SS.fit_transform(y_train_Tensile_Strength_MLP)  
y_test_Tensile_scale_MLP = SS.fit_transform(y_test_Tensile_Strength_MLP)


# Initialize the random grid

Random_Grid_Tensile = {"hidden_layer_sizes": [int(e) for e in np.arange(1,100)], 
                       "activation": ["relu", "tanh", "logistic"],
                       "solver": ["adam", "sgd", "lbfgs"], 
                       "alpha": [float(f) for f in np.logspace(0.0001,0.05)],
                       "learning_rate": ["constant", "adaptive"],
                       "max_iter": np.arange(1,10000,1)}


# Apply the RandomizedSearchCV

Random_Reg_Tensile_MLP = RandomizedSearchCV(MLP, Random_Grid_Tensile, n_jobs =- 1, cv = 5)

# Fit MLP using the optimized hyperparameters

Random_Reg_Tensile_MLP.fit(X_train_Tensile_scale_MLP, y_train_Tensile_scale_MLP.ravel())

Random_Reg_Tensile_MLP

RandomizedSearchCV(cv=5, estimator=MLPRegressor(), n_jobs=-1,
                   param_distributions={'activation': ['relu', 'tanh',
                                                       'logistic'],
                                        'alpha': [1.0002302850208247,
                                                  1.0025784544921414,
                                                  1.0049321365938482,
                                                  1.0072913442675548,
                                                  1.0096560904852534,
                                                  1.01202638824939,
                                                  1.014402250592935,
                                                  1.0167836905794556,
                                                  1.019170721303187,
                                                  1.0215633558891057,
                                                  1.023961607493,
                                                  1.026365489301543,
                                                  1....
                                                  1.0556560693917905,
                                                  1.0581343580333953,
                                                  1.0606184647769075,
                                                  1.0631084032810711,
                                                  1.065604187236696,
                                                  1.0681058303667321,
                                                  1.0706133464263468, ...],
                                        'hidden_layer_sizes': [1, 2, 3, 4, 5, 6,
                                                               7, 8, 9, 10, 11,
                                                               12, 13, 14, 15,
                                                               16, 17, 18, 19,
                                                               20, 21, 22, 23,
                                                               24, 25, 26, 27,
                                                               28, 29, 30, ...],
                                        'learning_rate': ['constant',
                                                          'adaptive'],
                                        'max_iter': array([   1,    2,    3, ..., 9997, 9998, 9999]),
                                        'solver': ['adam', 'sgd', 'lbfgs']})

# MLP best parameters (Tensile Strength)

Random_Reg_Tensile_MLP.best_params_

{'solver': 'lbfgs',
 'max_iter': 5572,
 'learning_rate': 'constant',
 'hidden_layer_sizes': 19,
 'alpha': 1.0287750145323662,
 'activation': 'relu'}

# MLP best estimator (Tensile Strength)

Random_Reg_Tensile_MLP.best_estimator_

MLPRegressor(alpha=1.0287750145323662, hidden_layer_sizes=19, max_iter=5572,
             solver='lbfgs')

# MLP tensile strength prediction

y_Random_Reg_Tensile_MLP_pred = Random_Reg_Tensile_MLP.predict(X_test_Tensile_scale_MLP)

print("R² Value = %.4f" % r2_score(SS.inverse_transform(y_test_Tensile_scale_MLP), SS.inverse_transform(y_Random_Reg_Tensile_MLP_pred)))
print("Mean Squared Error = %.2f" % mean_squared_error(y_test_Tensile_scale_MLP, y_Random_Reg_Tensile_MLP_pred))
print("Mean Squared Error (Non-Normalized) = %.2f" % mean_squared_error(SS.inverse_transform(y_test_Tensile_scale_MLP), SS.inverse_transform(y_Random_Reg_Tensile_MLP_pred)))

R² Value = 0.8444
Mean Squared Error = 0.16
Mean Squared Error (Non-Normalized) = 54577.92

4.11 Dataframe Representation of Predicted Output Tensile Strength Values (MLP) with Input Target Tensile Strength Values (MLP)¶

Back To Top

Once the predicted tensile strength values from the multilayer perceptron (MLP) model (y_Random_Reg_Tensile_MLP_pred) have been made according to (80:20) train-test split and the seed value of 14 for randomizing and shuffling with hyperparameters optimization from the RandomSearchCV function, the predicted results were added into a independent dataframe which contained the remaining 20% (y_test_Tensile_Strength_MLP) tensile strength values and the predicted (y_Random_Reg_Tensile_MLP_pred) tensile strength values in 2 seperate columns for visualisation of the predicted tensile strength values (y_Random_Reg_Tensile_MLP_pred) against the 20% (y_test_Tensile_Strength_MLP) actual tensile strength values.

The predicted tensile strength values (y_Random_Reg_Tensile_MLP_pred) when correlated with the test tensile strength values, have a R2 value of 0.8444 with a normalized mean squared error of 0.16 which is also generally good for tensile strength prediction as the correlations between the tensile strength with the input independent (X) variables defined by the ordinary least squared (OLS) module are generally good but not excellent. However, if all the elements atomic percentages (at%) and the encoded synthesis methods and microstructure are set as the input independent (X) variables, the R2 will definitely be worse than 0.8444 as it will incorporate the redundant correlations which have little to no effect on the prediction of the tensile strength and thereby, affect the predictions and produce bad R2 values. However, the non-normalized error is 54577.92 which is high but it should be noted that the prediction values are large and it is natural for the non-normalized error value to be somewhat large.

# Insert the predicted output tensile strength values with the input target tensile strength values

MLP_Tensile_Strength_Comparison = pd.DataFrame(SS.inverse_transform(y_test_Tensile_scale_MLP), columns = ["Tensile Strength Values MLP (y_test)"])
MLP_Tensile_Strength_Comparison.insert(1, "Tensile Strength Values MLP (y_pred)", SS.inverse_transform(y_Random_Reg_Tensile_MLP_pred))
MLP_Tensile_Strength_Comparison

4.12 Graphical Representation of Predicted Output Tensile Strength Values (MLP) with Input Target Tensile Strength Values (MLP)¶

Back To Top

This graphical representation was made to show the visualisation of the predicted tensile strength values from the hyperparameter optimized multilayer perceptron (MLP) model (y_Random_Reg_Tensile_MLP_pred) against the 20% test tensile strength values (y_test_Tensile_Strength_MLP). The blue scatter plot represents the correlation relationship between the predicted tensile strength values (y_Random_Reg_Tensile_MLP_pred) against the 20% test tensile strength values (y_test_Tensile_Strength_MLP).

The trendline was also calculated using a polynomial fit to produce the trendline of the blue scatter plot with its equation displayed. The R2 value produced from the multilayer perceptron (MLP) model and the R2 value calculated from the trendline were also displayed accordingly. The R2 value from the multilayer perceptron (MLP) model was 0.8444 while the one calculated from the trendline was 0.849.

Based on the comparison of the R2 values from both the multilayer perceptron (MLP) model and the one from the trendline, both are almost identical. It should be noted that there are some outliers present which is expected due to the complexity of the tensile strength prediction with respect to the indpendent input variables used according to the ordinary least squares (OLS) module. This is one of the challenges faced when predicting the tensile strength just like for yield strength prediction. However, based on the R2 value of 0.8444 from the multilayer perceptron (MLP) model and the indpendent input variables used according to the ordinary least squares (OLS) module, it can be considered to be a good prediction with room for improvement.

The unique property of the rectified linear activation function ("relu") is that it works in such a way that the linearity will lie with a range of values between zero and infinity. This is essential for the prediction of the tensile strength as tensile strength values are always positive and can never be negative. The range of positive values can lie anywhere within the positive scale towards very high positive values and the upper bound of infinity provides an endless possibility of predicting the positive values of the tensile strength with respect to the other optimized hyperparameters derived from the RandomizedSearchCV and the statistically significant input independent (X) variables. This phenomenon can clearly seen from the graphical representation.

# MLP Tensile Strength Graph


# Definition of x and y axes variables

Tensile_Strength_Plot_MLP = MLP_Tensile_Strength_Comparison.plot.scatter(x = "Tensile Strength Values MLP (y_test)", y = "Tensile Strength Values MLP (y_pred)", c="Blue", 
                                                                     figsize = (10, 7))


Legend_Tensile_Strength_MLP = plt.legend(["Tensile Strength Values MLP (y_pred) Vs. Tensile Strength Values MLP (y_test)"], title = None, bbox_to_anchor = (1, 0.9), 
                                       loc = "lower left")

plt.gca().add_artist(Legend_Tensile_Strength_MLP)


# Display trendline, respective equation and trendline R2 value

x_Tensile_Strength_MLP = MLP_Tensile_Strength_Comparison["Tensile Strength Values MLP (y_test)"]
y_Tensile_Strength_MLP = MLP_Tensile_Strength_Comparison["Tensile Strength Values MLP (y_pred)"]
z_Tensile_Strength_MLP = np.polyfit(x_Tensile_Strength_MLP, y_Tensile_Strength_MLP, 1)
p_Tensile_Strength_MLP = np.poly1d(z_Tensile_Strength_MLP)
y_hat_Tensile_Strength_MLP = np.poly1d(z_Tensile_Strength_MLP)(x_Tensile_Strength_MLP)

Text_Tensile_Strength_MLP = f"y = {z_Tensile_Strength_MLP[0]:0.3f}x{z_Tensile_Strength_MLP[1]:+0.3f}\nR² (Trendline) = {r2_score(y_Tensile_Strength_MLP, y_hat_Tensile_Strength_MLP):0.3f}"

Trendline_Tensile_Strength_MLP = plt.plot(x_Tensile_Strength_MLP, p(x_Tensile_Strength_MLP), c = "Red")

Legend_Reference_Tensile_Strength_MLP = plt.legend(["Data Trendline"], title = None, bbox_to_anchor = (1, 0.9), loc = "upper left")


# Plotting of MLP Tensile Strength Graph

plt.title("Graph of Predicted Output Tensile Strength Values MLP (y_pred) Vs. True Input Tensile Strength Values MLP (y_test)")
plt.xlabel("True Input Tensile Strength Values MLP (y_test)")
plt.ylabel("Predicted Output Tensile Strength Values MLP (y_pred)")

print("Tensile Strength (MLP Regression)")
print(" ")
print("R² Value = %.4f" % r2_score(SS.inverse_transform(y_test_Tensile_scale_MLP), SS.inverse_transform(y_Random_Reg_Tensile_MLP_pred)))
print("Mean Squared Error = %.2f" % mean_squared_error(y_test_Tensile_scale_MLP, y_Random_Reg_Tensile_MLP_pred))
print("Mean Squared Error (Non-Normalized) = %.2f" % mean_squared_error(SS.inverse_transform(y_test_Tensile_scale_MLP), SS.inverse_transform(y_Random_Reg_Tensile_MLP_pred)))
print(" ")
print(Text_Tensile_Strength_MLP)
print(" ")

plt.show()

Tensile Strength (MLP Regression)
 
R² Value = 0.8444
Mean Squared Error = 0.16
Mean Squared Error (Non-Normalized) = 54577.92
 
y = 0.902x+98.218
R² (Trendline) = 0.849

4.13 Elongation Best Parameters Selection (MLP)¶

Back To Top

After using the multilayer perceptron (MLP) model to predict the cost, yield strength and tensile strength the final major task is to predict the elongation. Compared with cost, the elongation is also not easy to predict as the predicted relationship may not be linear as with cost.

If all the elements atomic percentages (at%) columns together with the encoded synthesis methods and microstructure columns are set as the independent (X) variables and the elongation column is set as the dependent (y) variable which is to be predicted, the predictions can have a high chance to be inaccurate as some of the columns which have been set as the input columns have little to no correlations with the elongation and their redundant correlations will cause the multilayer perceptron (MLP) model to take them into consideration and produce bad predictions with bad R2 values together with the nature of the complex dataset. The redundant correlations can be seen and compared in the pearson correlations table in section 2.17.

The first step in predicting the elongation would be to do generalization just as for what was done for the yield strength and tensile strength predictions. Generalization, in this case, would be to implement the ordinary least squares (OLS) module to display all the columns that are to be used as the indpendent (X) input variables and set the dependent (y) variable to be the elongation column. Generalization will be able to measure the p-values which are the probability values. A p-value of less than 0.05 is statistically significant and it implies that it has significance to the dependent (y) variable which is the elongation and it rejects the null hypothesis. Any input variable with p-values larger than 0.05 will have little to no significance in the prediction of the elongation as it retains the null hypothesis.

Therefore, the ordinary least squares (OLS) module will be executed as many times as possible with the removel of the input independent (X) variable that produces the largest p-value each time until all the leftover input independent (X) variables have their respective p-values below the threshold value of 0.05. The elimination of all input independent (X) variables using the ordinary least squares (OLS) module, were executed below until the leftover input independent (X) variables have their respective p-values below the threshold value of 0.05. The elongation values have already be set to log values with base 10.

# Execute Ordinary Least Squares (OLS) module

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "Si(at%)", "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Re(at%)", "Ni(at%)", "O(at%)", 
                          "Y(at%)", "TiB2(at%)", "Test Temperature (°C)", "Synthesis Methods", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Log 10 (Elongation)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Synthesis Methods

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", "Ta(at%)",
                          "Si(at%)", "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Re(at%)", "Ni(at%)", "O(at%)", 
                          "Y(at%)", "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Log 10 (Elongation)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Ta

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", 
                          "Si(at%)", "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Re(at%)", "Ni(at%)", "O(at%)", 
                          "Y(at%)", "TiB2(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Log 10 (Elongation)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove TiB2

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", 
                          "Si(at%)", "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Re(at%)", "Ni(at%)", "O(at%)", 
                          "Y(at%)", "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Log 10 (Elongation)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Y

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "V(at%)", "Zr(at%)", "Hf(at%)", 
                          "Si(at%)", "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Re(at%)", "Ni(at%)", "O(at%)", 
                          "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Log 10 (Elongation)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove V

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "Zr(at%)", "Hf(at%)", 
                          "Si(at%)", "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Re(at%)", "Ni(at%)", "O(at%)", 
                          "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Log 10 (Elongation)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Re

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Mn(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "Zr(at%)", "Hf(at%)", 
                          "Si(at%)", "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Ni(at%)", "O(at%)", 
                          "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Log 10 (Elongation)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Mn

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "Zr(at%)", "Hf(at%)", 
                          "Si(at%)", "W(at%)", "C(at%)", "B(at%)", "Fe(at%)", "Ru(at%)", "Ni(at%)", "O(at%)", 
                          "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Log 10 (Elongation)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove B

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "Zr(at%)", "Hf(at%)", 
                          "Si(at%)", "W(at%)", "C(at%)", "Fe(at%)", "Ru(at%)", "Ni(at%)", "O(at%)", 
                          "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Log 10 (Elongation)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove O

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "Zr(at%)", "Hf(at%)", 
                          "Si(at%)", "W(at%)", "C(at%)", "Fe(at%)", "Ru(at%)", "Ni(at%)", 
                          "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Log 10 (Elongation)"], exog = X_opt).fit()
regressor_OLS.summary()

# Remove Ni

X_opt = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "Zr(at%)", "Hf(at%)", 
                          "Si(at%)", "W(at%)", "C(at%)", "Fe(at%)", "Ru(at%)", 
                          "Test Temperature (°C)", "Microstructure"]]

regressor_OLS = sm.OLS(endog = Encoded_Dataset_2["Log 10 (Elongation)"], exog = X_opt).fit()
regressor_OLS.summary()

4.14 Multilayer Perceptron (MLP) (Elongation)¶

Back To Top

Once the ordinary least squares module was implemented and the leftover input independent (X) variables with the respective p-values of less than the threshold value of 0.05 were determined, the leftover independent (X) variables were set as the input columns and the dependent (y) variable (elongation) was set as the output for prediction. The input independent (X) variable columns which are statistically significant in the prediction of the elongation are the titanium (Ti(at%)), niobium (Nb(at%)), chromium (Cr(at%)), molybdenum (Mo(at%)), zirconium (Zr(at%)), hafnium (Hf(at%)), silicon (Si(at%)), tungsten (W(at%)), carbon (C(at%)), iron (Fe(at%)), ruthenium (Ru(at%)), test temperature (Test Temperature (°C)) and the microstructure (Microstructure) columns.

The standard configuration for the multilayer perceptron (MLP) model was similar as with the one used for the cost, yield strength and tensile strength predictions with data scaling where the respective leftover independent (X) variables and the dependent (y) variable (elongation) had to be scaled with the StandardScalar module as the multilayer perceptron (MLP) model may not be able to process the data and predict correctly. The scaling would produce normalized smaller values with centering and the multilayer perceptron (MLP) model will be able to process them properly and predict correctly.

Just as with the multilayer perceptron (MLP) model configuration seen for the cost, yield strength and tensile strength predictions, the RandomizedSearchCV module was used to determine the best hyperparameters which are the hidden layer sizes, activation function, solver, alpha value, learning rate and maximum number of iterations.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the number of hidden layers it chose was 1 with 23 neurons. This was based on the nested cross validation of the already split 80:20 (80% train and 20% test) dataset.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the solver it chose for the deep gradient descent and backwards propagation to reduce the loss function to determine the smallest loss for the loss function (gradient descent local minima) was the log-linear "lbfgs" which is also known as the limited-memory-BFGS activation function. This solver is non-linear but the logarithmic values are linear.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the alpha value it chose was 1.11 which is already a logarithmic value derived from the range of 0.0001 to 0.05. The value is not too high and not too low but this value can be problem specific and usually a value slightly larger than 1 is commonly used. Therefore, it can be a little difficult to tell if the model has been underfitted or overfitted but based on the value selected by the RandomizedSearchCV function, it can be assumed that the model is just nice due to the value of alpha not being greater than 3.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the learning rate it chose was adaptive which is a type of learning rate where the updating of the weights after each successive deep gradient descent is adaptive with improvement in every successive loss function with every successive gradient descent with back propagation until the smallest loss function is reached (minima of the gradient descent curve) with the smallest amount of errors with convergence.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the maximum number of iterations it chose was 1589. This hyperparameter indicates how many forward propagations to impose through the summation of weights of the input feature variables in each layer of the neural network are transformed into the output which will be the predicted elongation. The more the number of times the loss function is calculated with differentiation to allow deep gradient descent to occur with the larger the number of times the errors contributing to the loss function are determined with backwards propagation, the larger the number of times those errors discovered with each iteration will be rectified with weights adjustments to improve the overall loss function with gradient descent and forward propagation with each iteration until the smallest loss function is reached (minima of the gradient descent curve) with the smallest amount of errors with convergence. The value of 1589 indicates that the smallest loss function is reached (minima of the gradient descent curve) with the smallest amount of errors with convergence after 1589 iterations.

Based on the hyperparameters chosen by the RandomizedSearchCV function, the activation function it chose was "tanh" which is the hyperbolic tangent scaled sigmoid activation function. This indicates that the prediction for the elongation may or may not be linear and since it chose the hyperbolic tangent scaled sigmoid activation function for the elongation prediction, the multilayer perceptron (MLP) model will work in such a way that the there can be a specified minimum and maximum range for the predicted output elongation values as the hyperbolic tangent scaled sigmoid activation function works with a scale from -1 to 1. The predicted relationship with the 20% test elongation values can also correlate in such a way that there will be specified minimum and maximum range when plotted on a graph.

The R2 value for the multilayer perceptron (MLP) model was 0.7959 with a normalized mean squared error of 0.20. This indicates that the multilayer perceptron (MLP) model is able to predict the elongation values using the input variables which have the most significant impact for the prediction based on the ordinary least squares (OLS) redundant input variables elimination as well as with the rest of the optimized hyperparameter selected by the RandomizedSearchCV function. The outcome will be used for the in-depth analysis during the graphical representation illustrated further, to produce the visualisation of how well the predicted elongation values from the multilayer perceptron (MLP) model (y_Random_Reg_Elongation_MLP_pred) correlate with the 20% (y_test_Elongation_MLP) elongation values as well as elaborate on how the hyperbolic tangent scaled sigmoid activation function contributes to the predicted output elongation values.

# MLP for Elongation

mySeed_MLP = np.random.seed(29)


# Define the MLP regressor

MLP = MLPRegressor(random_state = mySeed_MLP, shuffle = True)


# Define the independent (X) variable and dependent target (y) variable

X_Elongation_MLP, y_Elongation_MLP = Encoded_Dataset_2.loc[:, ["Ti(at%)", "Nb(at%)", "Cr(at%)", "Mo(at%)", "Zr(at%)", "Hf(at%)", "Si(at%)", "W(at%)", "C(at%)", "Fe(at%)", "Ru(at%)", "Test Temperature (°C)", "Microstructure"]], Encoded_Dataset_2["Log 10 (Elongation)"]


# Reshape the dependent (y) target variable into an array

y_reshape_Elongation_MLP = y_Elongation_MLP.values.reshape(-1, 1)


# Perform the train-test split

X_train_Elongation_MLP, X_test_Elongation_MLP, y_train_Elongation_MLP, y_test_Elongation_MLP = train_test_split(X_Elongation_MLP, y_reshape_Elongation_MLP, 
                                                                                                               train_size = 0.8, test_size = 0.2)

# Perform scaling and centering of data

SS = StandardScaler()

X_Elongation_scale_MLP = SS.fit_transform(X_Elongation_MLP) 
y_Elongation_scale_MLP = SS.fit_transform(y_reshape_Elongation_MLP)

X_train_Elongation_scale_MLP = SS.fit_transform(X_train_Elongation_MLP)  
X_test_Elongation_scale_MLP = SS.fit_transform(X_test_Elongation_MLP)
y_train_Elongation_scale_MLP = SS.fit_transform(y_train_Elongation_MLP)  
y_test_Elongation_scale_MLP = SS.fit_transform(y_test_Elongation_MLP)


# Initialize the random grid

Random_Grid_Elongation = {"hidden_layer_sizes": [int(g) for g in np.arange(1,100)], 
                          "activation": ["relu", "tanh", "logistic"],
                          "solver": ["adam", "sgd", "lbfgs"], 
                          "alpha": [float(h) for h in np.logspace(0.0001,0.05)],
                          "learning_rate": ["constant", "adaptive"],
                          "max_iter": np.arange(1,10000,1)}


# Apply the RandomizedSearchCV

Random_Reg_Elongation_MLP = RandomizedSearchCV(MLP, Random_Grid_Elongation, n_jobs =- 1, cv = 5)


# Fit MLP using the optimized hyperparameters

Random_Reg_Elongation_MLP.fit(X_train_Elongation_scale_MLP, y_train_Elongation_scale_MLP.ravel())

Random_Reg_Elongation_MLP

RandomizedSearchCV(cv=5, estimator=MLPRegressor(), n_jobs=-1,
                   param_distributions={'activation': ['relu', 'tanh',
                                                       'logistic'],
                                        'alpha': [1.0002302850208247,
                                                  1.0025784544921414,
                                                  1.0049321365938482,
                                                  1.0072913442675548,
                                                  1.0096560904852534,
                                                  1.01202638824939,
                                                  1.014402250592935,
                                                  1.0167836905794556,
                                                  1.019170721303187,
                                                  1.0215633558891057,
                                                  1.023961607493,
                                                  1.026365489301543,
                                                  1....
                                                  1.0556560693917905,
                                                  1.0581343580333953,
                                                  1.0606184647769075,
                                                  1.0631084032810711,
                                                  1.065604187236696,
                                                  1.0681058303667321,
                                                  1.0706133464263468, ...],
                                        'hidden_layer_sizes': [1, 2, 3, 4, 5, 6,
                                                               7, 8, 9, 10, 11,
                                                               12, 13, 14, 15,
                                                               16, 17, 18, 19,
                                                               20, 21, 22, 23,
                                                               24, 25, 26, 27,
                                                               28, 29, 30, ...],
                                        'learning_rate': ['constant',
                                                          'adaptive'],
                                        'max_iter': array([   1,    2,    3, ..., 9997, 9998, 9999]),
                                        'solver': ['adam', 'sgd', 'lbfgs']})

# MLP best parameters (Elongation)

Random_Reg_Elongation_MLP.best_params_

{'solver': 'lbfgs',
 'max_iter': 1589,
 'learning_rate': 'adaptive',
 'hidden_layer_sizes': 23,
 'alpha': 1.1089402918498994,
 'activation': 'tanh'}

# MLP best estimator (Elongation)

Random_Reg_Elongation_MLP.best_estimator_

MLPRegressor(activation='tanh', alpha=1.1089402918498994, hidden_layer_sizes=23,
             learning_rate='adaptive', max_iter=1589, solver='lbfgs')

# MLP elongation prediction

y_Random_Reg_Elongation_MLP_pred = Random_Reg_Elongation_MLP.predict(X_test_Elongation_scale_MLP)

print("R² Value = %.4f" % r2_score(SS.inverse_transform(y_test_Elongation_scale_MLP), SS.inverse_transform(y_Random_Reg_Elongation_MLP_pred)))
print("Mean Squared Error = %.2f" % mean_squared_error(y_test_Elongation_scale_MLP, y_Random_Reg_Elongation_MLP_pred))
print("Mean Squared Error (Non-Normalized) = %.2f" % mean_squared_error(SS.inverse_transform(y_test_Elongation_scale_MLP), SS.inverse_transform(y_Random_Reg_Elongation_MLP_pred)))

R² Value = 0.7959
Mean Squared Error = 0.20
Mean Squared Error (Non-Normalized) = 0.09

4.15 Dataframe Representation of Predicted Output Elongation Values (MLP) with Input Target Elongation Values (MLP)¶

Back To Top

Once the predicted elongation values from the multilayer perceptron (MLP) model (y_Random_Reg_Elongation_MLP_pred) have been made according to (80:20) train-test split and the seed value of 29 for randomizing and shuffling with hyperparameters optimization from the RandomSearchCV function, the predicted results were added into a independent dataframe which contained the remaining 20% (y_test_Elongation_MLP) elongation values and the predicted (y_Random_Reg_Elongation_MLP_pred) elongation values in 2 seperate columns for visualisation of the predicted elongation values (y_Random_Reg_Elongation_MLP_pred) against the 20% (y_test_Elongation_MLP) actual elongation values.

The predicted elongation values (y_Random_Reg_Elongation_MLP_pred) when correlated with the test elongation values, have a R2 value of 0.7959 with a normalized mean squared error of 0.20 which is also generally good for elongation prediction as the correlations between the elongation with the input independent (X) variables defined by the ordinary least squared (OLS) module are generally good but not excellent. However, if all the elements atomic percentages (at%) and the encoded synthesis methods and microstructure are set as the input independent (X) variables, the R2 will definitely be worse than 0.7959 as it will incorporate the redundant correlations which have little to no effect on the prediction of the elongation and thereby, affect the predictions and produce bad R2 values. However, the non-normalized error is 0.09 which is actually good as well and this can be a good indication that the elongation values were generally well predicted with the multilayer perceptron (MLP) model due to very low mean squared errors for both the normalized and non-normalized variations.

# Insert the predicted output elongation values with the input target elongation values

MLP_Elongation_Comparison = pd.DataFrame(SS.inverse_transform(y_test_Elongation_scale_MLP), columns = ["Log 10 Elongation Values MLP (y_test)"])
MLP_Elongation_Comparison.insert(1, "Log 10 Elongation Values MLP (y_pred)", SS.inverse_transform(y_Random_Reg_Elongation_MLP_pred))
MLP_Elongation_Comparison

4.16 Graphical Representation of Predicted Output Elongation Values (MLP) with Input Target Elongation Values (MLP)¶

Back To Top

This graphical representation was made to show the visualisation of the predicted elongation values from the hyperparameter optimized multilayer perceptron (MLP) model (y_Random_Reg_Elongation_MLP_pred) against the 20% test elongation values (y_test_Elongation_MLP). The blue scatter plot represents the correlation relationship between the predicted elongation values (y_Random_Reg_Elongation_MLP_pred) against the 20% test elongation values (y_test_Elongation_MLP).

The trendline was also calculated using a polynomial fit to produce the trendline of the blue scatter plot with its equation displayed. The R2 value produced from the multilayer perceptron (MLP) model and the R2 value calculated from the trendline were also displayed accordingly. The R2 value from the multilayer perceptron (MLP) model was 0.7959 while the one calculated from the trendline was 0.814.

Based on the comparison of the R2 values from both the multilayer perceptron (MLP) model and the one from the trendline, both are very similar. It should be noted that there are some outliers present which is expected due to the complexity of the elongation prediction with respect to the indpendent input variables used according to the ordinary least squares (OLS) module. This is one of the challenges faced when predicting the elongation just like for yield strength prediction. However, based on the R2 value of 0.7959 from the multilayer perceptron (MLP) model and the indpendent input variables used according to the ordinary least squares (OLS) module, it can be considered to be a good prediction with room for improvement.

The unique factor for the elongation prediction was the fact that the logarithmic values for the elogations lie between a specified range. As they lie within the specified range, the RandomizedSearchCV function chose the hyperbolic tangent scaled sigmoid activation function ("tanh") as this activation function works with a scale from -1 to 1 which coincides with the specified range of the logarithmic elongation values where they can lie between the negative and positive range and the multilayer perceptron (MLP) model worked in such a way that the there was a specified minimum and maximum range for the predicted output elongation values, in which in this case, based on the graphical representation, was -2.5 to 0.5 with some linearity. The 20% test logarithmic elongation values (y_test_Elongation_MLP) also had a range of -2.5 to 0.0. This phenomenon can be clearly seen from the graphical representation.

# MLP Elongation Graph


# Definition of x and y axes variables

Elongation_Plot_MLP = MLP_Elongation_Comparison.plot.scatter(x = "Log 10 Elongation Values MLP (y_test)", y = "Log 10 Elongation Values MLP (y_pred)", c="Blue", 
                                                                     figsize = (10, 7))


Legend_Elongation_MLP = plt.legend(["Elongation Values MLP (y_pred) Vs. Elongation Values MLP (y_test)"], title = None, bbox_to_anchor = (1, 0.9), 
                                       loc = "lower left")

plt.gca().add_artist(Legend_Elongation_MLP)


# Display trendline, respective equation and trendline R2 value

x_Elongation_MLP = MLP_Elongation_Comparison["Log 10 Elongation Values MLP (y_test)"]
y_Elongation_MLP = MLP_Elongation_Comparison["Log 10 Elongation Values MLP (y_pred)"]
z_Elongation_MLP = np.polyfit(x_Elongation_MLP, y_Elongation_MLP, 1)
p_Elongation_MLP = np.poly1d(z_Elongation_MLP)
y_hat_Elongation_MLP = np.poly1d(z_Elongation_MLP)(x_Elongation_MLP)

Text_Elongation_MLP = f"y = {z_Elongation_MLP[0]:0.3f}x{z_Elongation_MLP[1]:+0.3f}\nR² (Trendline) = {r2_score(y_Elongation_MLP, y_hat_Elongation_MLP):0.3f}"

Trendline_Elongation_MLP = plt.plot(x_Elongation_MLP, p(x_Elongation_MLP), c = "Red")

Legend_Reference_Elongation_MLP = plt.legend(["Data Trendline"], title = None, bbox_to_anchor = (1, 0.9), loc = "upper left")


# Plotting of MLP Elongation Graph

plt.title("Graph of Predicted Output Log 10 (Elongation) Values MLP (y_pred) Vs. True Input Elongation Values MLP (y_test)")
plt.xlabel("True Input Log 10 (Elongation) Values MLP (y_test)")
plt.ylabel("Predicted Output Elongation Values MLP (y_pred)")

print("Elongation (Log 10) (MLP Regression)")
print(" ")
print("R² Value = %.4f" % r2_score(SS.inverse_transform(y_test_Elongation_scale_MLP), SS.inverse_transform(y_Random_Reg_Elongation_MLP_pred)))
print("Mean Squared Error = %.2f" % mean_squared_error(y_test_Elongation_scale_MLP, y_Random_Reg_Elongation_MLP_pred))
print("Mean Squared Error (Non-Normalized) = %.2f" % mean_squared_error(SS.inverse_transform(y_test_Elongation_scale_MLP), SS.inverse_transform(y_Random_Reg_Elongation_MLP_pred)))
print(" ")
print(Text_Elongation_MLP)
print(" ")

plt.show()

Elongation (Log 10) (MLP Regression)
 
R² Value = 0.7959
Mean Squared Error = 0.20
Mean Squared Error (Non-Normalized) = 0.09
 
y = 0.785x-0.397
R² (Trendline) = 0.814

5. Linear Regression Model Vs. Multilayer Perceptron (MLP) Vs. Alchemite¶

5.1 Alchemite¶

Back To Top

Apart from the linear regression model and the multilayer perceptron (MLP) model in python, Alchemite was also used to predict the output costs in a similar way that the linear regression model did. Alchemite was also used to predict the yield strength,, tensile strength and elongation (Log 10). However, there are several differences which will be discussed. Alchemite is a neural network created by Intellegens in UK and it is specifically used for property predictions of different materials as well as for drug delivery predictive purposes. It uses various forms of algorithms which are executed by JSON files for predictions.

The similarities between python's linear regression model and Alchemite is that the standard data splitting into the training sets and test sets, randomizations and shuffling with the defining of the independent (X) and dependent (y) variables before doing any predictions, are common to both machine learning methods. However, Alchemite has parameters such as gradient tilting, gradient directions, iteration layers, extra validation etc. which are features that are similar to the multilayer perceptron (MLP) model's avtivation function, solver, maximum iterations, hidden layer sizes and alpha etc. as they are both neural networks.

Alchemite’s parameters can also be enhanced with python's hyperparameter optimization code which can run at least a 200 test runs which imports the operating system and it is executed through command lines to produce the best hyperparameters for any specific training and testing datasets through various defined automated fitting mechanisms.

The test runs are automated trials of data fitting and the 5-fold nested cross validation, hold-out cross validation, 1-time fixed 80:20 split with several gradient tilting functions, nodes definition functions and Alchemite-specific extracted files etc. will be utilised to determine which of these can provide the best combinations for the model fitting. Once the best combinations are determined with the appropriate fitting model through the 200 test runs of fitting, the corresponding hyperparameters will be the optimized hyperparameters and they will be extracted and written in the JSON file code format automatically.

The resultant optimized hyperparameters can then be used from the JSON file to specify the instructions for Alchemite to run with while incorporating the training and testing datasets. The hyperparameter optimization python code could not be shared in this coursework as permission could not be granted by Alchemite for free-sharing purposes. However, permission has been granted by Alchemite to share a sample JSON file code that Alchemite uses. The sample JSON file code includes some descriptions of the several instructions seen which are executed through command lines in Alchemite to determine the R2 values.

5.2 Alchemite JSON File Code¶

Back To Top

Sample JSON code used by Alchemite for property predictions

Training file and training task specifications

{
    "training": {
    "input nodes":25, (input nodes define the input columns for the independent X variable)
    "descriptor columns":25, (descriptor columns also define of the input columns for the independent X variable)
    "fraction divisions tried":1

Tasks to be executed

},
"tasks":["train","validate"], (train is the training task (80% train) and validate is the validation task (20% test)

Network parameters

"network": {
    "distribution samples": 4000, (number of times the model will iterate to produce optimal prediction results)
    "iteration layers": 1, (used to compensate for missing values)
    "data for intercept": 2,
    "data for gradient": 5,   (used for gradient tilting to produce better fitting)
    "gradient directions": 2, (used for gradient tilting to produce better fitting)
    "data in gradient": 1,
    "data in intercept": 1,
},

Extra validation task

"validation":{
    "coefficient of determination elements":true
}

}

5.3 Linear Regression Model Vs. Multilayer Perceptron (MLP) Vs. Alchemite (Cost Prediction)¶

Back To Top

For the cost predictions, both the linear regression model and Alchemite provide similar R2 values although there is a slight difference. However, it can be noted that both methods have R2 values which are extremely close to 1 (Linear regression model (R2 = 0.9989)) (Alchemite (R2 = 0.9751)). When compared with the multilayer perceptron (MLP) model, the and the multilayer perceptron (MLP) model is able to also have a near perfect match with the linear regression and Alchemite. This can indicate that all 3 machine learning methods can provide excellent predictions.

# R2 values comparison from Alchemite, linear regression and MLP (Cost predictions)


# Display results from linear regression and MLP (mySeed = 1234555) (Best result)

Result_Main_1 = Image.open(r"./Best_Result.png")
Result_Main_2 = Image.open(r"./CostMLP.png")


print("Linear Regression Model Best Results (Seed Value = 1234555)")
plt.imshow(Result_Main_1)
plt.show()
print(" ")


print("Multilayer Perceptron (MLP) Model Results (Seed Value = 1234555)")
plt.imshow(Result_Main_2)
plt.show()
print(" ")


# Display results from Alchemite

Alchemite_Cost_Linear = Image.open(r"./Alchemite.png")

print("Alchemite Results (Only Cost)")
plt.imshow(Alchemite_Cost_Linear)
plt.show()
print(" ")

Linear Regression Model Best Results (Seed Value = 1234555)

 
Multilayer Perceptron (MLP) Model Results (Seed Value = 1234555)

 
Alchemite Results (Only Cost)

5.4 Multilayer Perceptron (MLP) Vs. Alchemite (Yield Strength Prediction)¶

Back To Top

For the yield strength predictions, both the multilayer perceptron (MLP) model and Alchemite provide somewhat similar R2 values (R2 (MLP) = 0.8394 and R2 (Alchemite) = 0.7218) although there is a slight difference. As the multilayer perceptron (MLP) model was trained with the optimized hyperparameters from the RandomSearchCV with the input variables selected by the ordinary least squared (OLS) module, these configurations may not have been applied with Alchemite although there may be some similarity between both neural networks for the prediction of the yield strength.

Alchemite also uses deep gradient descent to improve the loss function through backwards propagation although the type of solver, alpha values and learning rates, hidden layer sizes and activation function may have been different with 200 fixed iterations for the hyperparameter optimization which was based on Alchemite's hyperparameter optimization python script. This could be one of the major reasons why the difference between both R2 values have a slight difference of 0.12 which can be considered to be quite good in terms of predictive abilities as both neural networks are able to predict the yield strength in a similar manner.

The error in uncertainties in Alchemite is the equivalent to the multilayer perceptron (MLP) model's mean squared error and it has a value of 0.073 (normalized) while the multilayer perceptron (MLP) model's mean squared error (normalized) is 0.28. Although there is a slightly higher normalized mean squared error for the multilayer perceptron (MLP) model, the overall combination of the respective R2 values combined with the respective mean squared errors for both models are very much similar although different configurations may have been used, indicating that both models can predict the yield strength based on their individual configurations and optimized hyperparameters overall.

# R2 values comparison from Alchemite and MLP (yield strength predictions)


# Display results from MLP (mySeed = 20) (Best result)

Result_Main_1 = Image.open(r"./YieldMLP.png")

print("Multilayer Perceptron (MLP) Model Results (Seed Value = 20)")
plt.imshow(Result_Main_1)
plt.show()
print(" ")


# Display results from Alchemite

Alchemite_Yield = Image.open(r"./Alchemite2.png")

print("Alchemite Results (Column 24 = Yield Strength)")
plt.imshow(Alchemite_Yield)
plt.show()
print(" ")

Multilayer Perceptron (MLP) Model Results (Seed Value = 20)

 
Alchemite Results (Column 24 = Yield Strength)

5.5 Multilayer Perceptron (MLP) Vs. Alchemite (Tensile Strength Prediction)¶

Back To Top

For the tensile strength predictions, both the multilayer perceptron (MLP) model and Alchemite provide very similar R2 values with a very small difference of 0.033. This can indicate that both machine learning methods can provide very good predictions for the tensile strength and the near perfect match of the R2 values of both models (R2 (MLP) = 0.8444 and R2 (Alchemite) = 0.8115) indicates that their individual configurations with their individual optimized hyperparameters are capable of predicting the tensile strength even though there were differences between the type of solver, alpha values and learning rates, hidden layer sizes, activation function and the maximum number of iterations to reach convergence with minimal loss function errors. Thes tensile strength predictions were slightly better than the yield strength predictions at least for the multilayer perceptron (MLP) model.

The error in uncertainties in Alchemite is the equivalent to the multilayer perceptron (MLP) model's mean squared error and it has a value of 0.083 (normalized) while the multilayer perceptron (MLP) model's mean squared error (normalized) is 0.16. Although there is a slightly higher normalized mean squared error for the multilayer perceptron (MLP) model, the overall combination of the respective R2 values combined with the respective mean squared errors for both models are very much similar although different configurations have been used, indicating that both models can predict the yield strength based on their individual configurations and optimized hyperparameters overall.

# R2 values comparison from Alchemite and MLP (tensile strength predictions)


# Display results from MLP (mySeed = 14) (Best result)

Result_Main_1 = Image.open(r"./TensileMLP.png")

print("Linear Regression Model Best Results (Seed Value = 14)")
plt.imshow(Result_Main_1)
plt.show()
print(" ")


# Display results from Alchemite

Alchemite_Tensile = Image.open(r"./Alchemite2.png")

print("Alchemite Results (Column 25 = Tensile Strength)")
plt.imshow(Alchemite_Tensile)
plt.show()
print(" ")

Linear Regression Model Best Results (Seed Value = 14)

 
Alchemite Results (Column 25 = Tensile Strength)

5.6 Multilayer Perceptron (MLP) Vs. Alchemite (Elongation Prediction)¶

Back To Top

For the elongation predictions, only the multilayer perceptron (MLP) model was able to provide a good prediction with a R2 value of 0.7959. The R2 value attained by Alchemite was 0.4467 which is not acceptable as the R2 value is considered to be very low and the difference between both R2 values is 0.3492 which is also unacceptable.

Alchemite was therefore, unable to produce similar results as with the multilayer perceptron (MLP) model. This could be possible due to the configuration of Alchemite with its hyperparameter optimization. Although it is not possible to determine what the exact pipeline flow of tasks are due to Alchemite being run using command lines, it can be deduced that some of its hyperparameters that were optimized may not have been favourable in the prediction of the elongation values compared to the multilayer perceptron (MLP) model. This could also be further enhanced by the fact that the number of iterations were fixed to 200 for Alchemite and a convergence could not have been reached with the backwards propagation with deep gradient descent the improval of the loss function (local minima for loss function could not have been reached).

The error in uncertainties in Alchemite is the equivalent to the multilayer perceptron (MLP) model's mean squared error and it has a value of 0.1527 (normalized) while the multilayer perceptron (MLP) model's mean squared error (normalized) is 0.20. Although there is a slightly higher normalized mean squared error for the multilayer perceptron (MLP) model, both mean squared values are insignificant in this case as the R2 values are very different from each other with Alchemite having a very low R2 value of 0.3492 which is not feasible. Alchemite's inability to properly predict the elongation values can be improved by further fine tuning its hyperparameters and increasing the number if iterations to reach convergence through the deep gradient descent.

# R2 values comparison from Alchemite and MLP (elongation predictions)


# Display results from MLP (mySeed = 29) (Best result)

Result_Main_1 = Image.open(r"./ElongationMLP.png")

print("Linear Regression Model Best Results (Seed Value = 29)")
plt.imshow(Result_Main_1)
plt.show()
print(" ")


# Display results from Alchemite

Alchemite_Elongation = Image.open(r"./Alchemite2.png")

print("Alchemite Results (Column 26 = Elongation (Log 10))")
plt.imshow(Alchemite_Elongation)
plt.show()
print(" ")

Linear Regression Model Best Results (Seed Value = 29)

 
Alchemite Results (Column 26 = Elongation (Log 10))

5.7 Methodology Principle For Multilayer Perceptron (MLP) Configuration and Alchemite¶

Back To Top

The overall methodology used for the predictions of the cost, yield strength, tensile strength and elongation (Log 10) properties was based on the principle of using existing properties to predict a new set of properties. To predict the cost, the elements atomic percentages (at%) was the essential components needed as they were converted to weight percentages (wt%) and those weight percentages (wt%) were used to calculate the cost using the atomic weights of every individual element. As the elements weight percentages (wt%) were directly used for the calculation of the costs which contributed to the cost training (y_train) and cost testing (y_test) data, the predicted cost values had a near perfect linear fit with cost testing (y_test) values regardless of whether the linear regression model or the multilayer perceptron (MLP) model was used.

This was not the same for the prediction of the yield strength, tensile strength and the elongation values. This was because neither of the 3 mechanical properties for the training and testing components were calculated using the elements percentages properties or using the encoded categorical variables (synthesis methods and microstructure). All values of the 3 mechanical properties which contributed to their individual training and testing datasets were taken from research papers, journals and textbooks. This implies that the values for all 3 mechanical properties were actually independent from the elements percentages properties and the encoded categorical variables (synthesis methods and microstructure). This is also why it will be a challenge to predict all 3 properties with the use of the elements percentages properties and the encoded categorical variables (synthesis methods and microstructure).

However, it should be noted that none of the values for all 3 mechanical properties were used in the ordinary least squares (OLS) module in order to determine the best individual input columns for predicting one another. This was because, to determine any of the 3 mechanical properties, they must always be the dependent (y) variable and not part of the independent (X) variable even though any of the 3 mechanical properties can very well have excellent pearson correlations with each other. An example would be the pearson correlation between the yield strength and tensile strength which was the strongest among all other variables although they were not set to be part of the independent (X) variables in the ordinary least squares (OLS) module to determine the best individual input columns.

This was because, to design a new material, the principle is such that the mechanical properties such as yield strength, tensile strength and elongations will be unknown as the material has not been synthesized yet. The basic premise would be to try and predict what the properties will be like by using the variables that can be known and will contribute to the creation of those 3 mechanical properties. This principle and premise was implemented in the ordinary least squares (OLS) module to determine the best individual input columns by not including any of the columns that contained the values of the 3 mechanical properties which would contribute to their individual training and testing datasets. This was one of the reasons why the R2 values for all 3 mechanical properties could not get pass 0.9 and there are higher chances of outliers to appear as seen in the graphical representations as some of the strongest correlations which can lie between any of the 3 mechanical properties to be predicted (yield strength and tensile strength perarson correlation being the strongest) were not considered as input columns in the ordinary least squares (OLS) module to determine the best individual input columns. The overall methodology was also implemented in Alchemite for the prediction of all 3 mechanical properties.

6. Limitations and Challenges¶

Back To Top

While doing this courswork, there were several challenges seen. The first challenge was to determine on how to determine the hyperparameters and what was the purpose of each hyperparameter. This was important as the optimized hyperparameters based on the dataset and neural network used, will determine if the predictions will be accurate or not. After some research, it was determined that the most important hyperparameters for the multilayer perceptron (MLP) neural network model can be optimized with either the GridSearchCV or RandomizedSearchCV.

Ultimately, RandomizedSearchCV was used as the number of features were large, complex and there was no way to tell what the range of hyperparameters to specify will be which is seen in GridSearchCV where it performs a gridsearch based on specified range of values for the hyperparameters to optimize and it is more useful for smaller, less complex datasets. Therefore, RandomizedSearchCV was used to search through a large space and determine the random factors which would contribute to the optimization of the hyperparameters.

The second challenge was not being to reach convergence with deep gradient descent with the lowest loss function (minima of gradient descent curve). This was because the solver could not perform the deep gradient descent with backpropagation to determine the smallest loss function error with the range of the number of maximum iterations defined. Therefore, to address this problem, the number of iterations was increased to 10000 which could help reach the local minima with the smallest loss function error with the chosen solver for gradient descent and backpropagation.

The third challenge was to determine why the R2 values for the predictions of the 3 mechanical properties were not good initially. This was due to the use of all columns that were defined as the independent (X) columns and whether if each input column was statistically significant or not, was not considered. This is where the ordinary least squares (OLS) module was implemented and the statistically significant columns (p-value < 0.05) for the predictions of each of the 3 mechanical properties, were retained with backwards elimination of the input columns that had no statistical significance (p-value > 0.05) in the prediction of any of the 3 mechanical properties. This procedure had a significant improvement on the prediction of the 3 mechanical properties and the R2 values were much better.

The fourth challenge was to check if the "identity" activation function was needed for the prediction of the 3 mechanical properties. The "identity" activation function is essentially the very basic linear function where the range is y=x and it can reach infinity for both axes and also be negative. The yield strength and tensile strength was impossible to be negative which is why it was not applicable. The elongation values were already logarithmic and there was a high chance that the activation function chosen will also not be linear for the prediction of the logarithmic elongation values as logarithmic relationships plotted a graph are non-linear in nature. The identity" activation function was however, applicable for the cost prediction as it was evident that the cost prediction relationship was linear between the predicted cost values and the test cost values. It was also assumed that the relationship may not be linear for the other 3 mechanical properties as well. Therefore, the "identity" activation function was not defeined as one of the activation functions to choose from for the RandomizedSearchCV for the 3 mechanical properties.

7. Conclusion¶

Back To Top

Overall, the multilayer perceptron (MLP) model was able to predict the cost, yield strength, tensile strength and elongation values based on the optimized hyperparameters determined from the RandomizedSearchCV module and the statistically significant (p-value < 0.05) input variables determined from the ordinary least squares (OLS) module. The R2 value for the yield strength prediction was 0.7218. The R2 value or the tensile strength prediction was 0.8444. The R2 value for the elongation prediction was 0.7959. The R2 values were not expected to rise above 0.9 as the strongest correlations between any of the 3 mechanical properties (yield strength, tensile strength and elongation) were not taken into consideration during the elimination of the statistically insignificant (p-value > 0.05) input variables.

This was due to the principle that the 3 mechanical properties cannot exist if a new material has not yet been synthesized and it is not viable to use the values of the already existing mechanical properties to predict other unknown mechanical properties. The viable approach will be to follow the principle that mechanical properties can only exist if a material has been synthesized and therefore, the factors that will contribute to the creating of mechanical properties will be the compositions of the elements present, synthesis methods and the microstructure. The values for the 3 mechanical properties will only be used as the training and testing values to train the multilayer perceptron (MLP) model and allow it to predict the new values for the respective mechanical properties and compare them with the test values of the respective mechanical properties.

When compared with the predictions done by Alchemite, it was seen that the predictions for the yield strength and tensile strength were generally good with small differences between the R2 values of the multilayer perceptron (MLP) model and Alchemite. However, Alchemite was not able to properly predict the elongation values compared to the multilayer perceptron (MLP) model. This could be because the maximum number of iterations set in Alchemite was 200 and it was possible that convergence could not have been reached with deep gradient descent to reduce the loss function error to the smallest amount (minima of the gradient descent curve). One way this problem can be rectified is by increasing the number of iterations until convergence is reached and the R2 value based on the new predictions can improve. Another method could be to further optimize Alchemite's other hyperparameters as well.

8. Future Work¶

Back To Top

One such task for the future would be to expand the dataset with more rows of more variations of the (γ) and ((α2) + (γ)) titanium alumimides with more data in the input columns such as elements atomic compositions, synthesis methods and microstructures. This could potentially improve the training of the neural network model regardless of it being the multilayer perceptron (MLP) model or Alchemite. Once there is more data present, the hyperparameters for both neural networks can be optimized until the best variables for their respective hyperparameters are chosen.

The ordinary least squares (OLS) module can then be applied for the multilayer perceptron (MLP) model to isolate the statistically significant input variables that have an impact on the predictions of the 3 mechanical properties (yield strength, tensile strength and elongation). The number of iterations for Alchemite can also be increased to achieve convergence through gradient descent to minimise the loss error function at least for the prediction of elongation.

Once both neural networks have significant predictions with good R2 values of at least above 0.7, particle swarm optimization can be implemented onto the predicted results from both neural networks for all 3 mechanical properties as well as for the cost, to further improve the predicted values to determine what the best combination of the cost and the 3 mechanical properties can be through the particle swarm optimization.

If the best combination or combinations of the newly predicted output mechanical properties through the particle swarm optimization is/are feasible for use in high temperature sections of the aircraft engines and can be made into reality with the synthesis of the corresponding new (γ) and (α2 + γ) titanium alloy/alloys based on the current processing methods available, the artificial neural networks multilayer perceptron (MLP), Alchemite and particle swarm optimization would have successfully incorporated a good and suitable predictive model to predict the material properties for new (γ) and (α2 + γ) titanium aluminide alloy/alloys for high temperature aerospace applications which would in turn, fulfil the final outcome of the actual project.

9. References¶

Back To Top

Intellegens Chief Technical Officer Dr Gareth Conduit

Institute of High-Performance Computing (IHPC) Project Manager Dr. Mark Hyunpong Jhon

Nanyang Technological University Project Manager Dr. Raju V. Ramanujan

Wu, X., 2006. Review of alloy and process development of TiAl alloys. Intermetallics, 14(10-11), pp.1114-1122.

Rodriguez, G., Bowen, J., Zelzer, M. and Stamboulis, A., 2020. Selective modification of Ti6Al4V surfaces for biomedical applications. RSC Advances, 10(30), pp.17642-17652.

McBride, J., Malinov, S. and Sha, W., 2004. Modelling tensile properties of gamma-based titanium aluminides using artificial neural network. Materials Science and Engineering: A, 384(1-2), pp.129-137.

Kim, Y., 1995. Gamma Titanium Aluminides: Their Status And Future.

Casadebaigt, A., Hugues, J. and Monceau, D., 2020. High temperature oxidation and embrittlement at 500–600 °C of Ti-6Al-4V alloy fabricated by Laser and Electron Beam Melting. Corrosion Science, 175, p.108875.

Conduit, B., Illston, T., Baker, S., Duggappa, D., Harding, S., Stone, H. and Conduit, G., 2019. Probabilistic neural network identification of an alloy for direct laser deposition. Materials & Design, 168, p.107644.

Franzén, S. and Karlsson, J., 2010. Γ-Titanium Aluminide Manufactured By Electron Beam Melting.

Froes, F. and Boyer, R., n.d. Additive Manufacturing For The Aerospace Industry.

Ma, R., Liu, Z., Wang, W., Xu, G. and Wang, W., 2020. Microstructures and mechanical properties of Ti6Al4V-Ti48Al2Cr2Nb alloys fabricated by laser melting deposition of powder mixtures. Materials Characterization, 164, p.110321.

Malinov, S. and Sha, W., 2004. Application of artificial neural networks for modelling correlations in titanium alloys. Materials Science and Engineering: A, 365(1-2), pp.202-211.

Sha, W., 2006. The evolution of microstructure during the processing of gamma Ti−Al alloys. JOM, 58(9), pp.64-66.

Sarkar, S., Mitra, S. and Bhattacharyya, B., 2005. Parametric optimisation of wire electrical discharge machining of γ titanium aluminide alloy through an artificial neural network model. The International Journal of Advanced Manufacturing Technology, 27(5-6), pp.501-508.

Tkachenko, R., Duriagina, Z., Lemishka, I., Izonin, I. and Trostianchyn, A., 2018. Development of machine learning method of titanium alloy properties identification in additive technologies. Eastern-European Journal of Enterprise Technologies, 3(12 (93), pp.23-31.

Gupta, R., Mehta, R., Agarwala, V., Pant, B. and Sinha, P., 2011. Study on Ductility of Ti Aluminide Using Artificial Neural Network. International Journal of Aerospace Engineering, 2011, pp.1-7.

Kim, J., Lee, Y., Kim, Y. and Lee, C., 2007. High Temperature Deformation Behavior of Beta-Gamma TiAl Alloy. Materials Science Forum, 539-543, pp.1531-1536.

Emmert-Streib, F. and Dehmer, M., 2019. Evaluation of Regression Models: Model Assessment, Model Selection and Generalization Error. Machine Learning and Knowledge Extraction, 1(1), pp.521-551.

Hsieh, P., Horng, D. and Chang, H., 2020. International Expansion Selection Model by Machine Learning—A Proprietary Model. The Computer Journal,.

Dzogbewu, T., 2020. Additive manufacturing of TiAl-based alloys. Manufacturing Review, 7, p.35.

En.wikipedia.org. 2021. Prices of chemical elements - Wikipedia. [online] Available at: https://en.wikipedia.org/wiki/Prices_of_chemical_elements [Accessed 2 July 2021].

University of London VLE lecture slides

Data Programming in python Coursework 1

	Yield Strength (MPa)	Tensile Strength (MPa)	Log 10 (Elongation)	Cost (USD/kg)
Ti (at%)	-0.457113	-0.498954	-0.016347	0.145384
Al (at%)	-0.194099	-0.150853	-0.097998	-0.054109
Mn (at%)	-0.047556	-0.049060	-0.002170	-0.027250
Nb (at%)	0.417353	0.372250	0.072625	-0.061519
Cr (at%)	0.220100	0.280381	-0.045341	-0.105853
Mo (at%)	0.135609	0.113770	0.148050	-0.064858
V (at%)	-0.052063	-0.070319	-0.048741	0.000624
Zr (at%)	0.061385	0.014106	-0.184026	-0.007942
Hf (at%)	0.046882	-0.013382	-0.163975	0.150980
Ta (at%)	0.029470	0.026132	-0.003671	0.009312
Si (at%)	-0.050968	-0.082802	-0.176009	0.016875
W (at%)	-0.062591	-0.090345	-0.123881	-0.047836
C (at%)	0.027612	0.146728	0.311001	-0.025702
B (at%)	0.115686	0.093366	0.136521	-0.031122
Fe (at%)	0.039772	0.220198	0.224545	-0.045456
Ru (at%)	-0.034867	-0.033957	0.137365	0.980318
Re (at%)	-0.051391	-0.033370	-0.018585	0.045287
Ni (at%)	-0.037551	-0.039240	-0.119528	-0.019003
O (at%)	-0.103533	-0.115270	0.021761	-0.037952
Y (at%)	-0.040350	-0.046275	-0.013646	0.004814
TiB2 (at%)	0.019463	-0.001397	0.010183	-0.011712
Test Temperature (°C)	-0.279728	-0.251764	0.572472	0.147966
Synthesis Methods	0.085640	-0.031307	-0.058594	-0.094777
Microstructure	0.138127	0.195047	0.080017	0.094702
Yield Strength (MPa)	1.000000	0.867486	0.093184	-0.001824
Tensile Strength (MPa)	0.867486	1.000000	0.261277	-0.015554
Log 10 (Elongation)	0.093184	0.261277	1.000000	0.111771
Cost (USD/kg)	-0.001824	-0.015554	0.111771	1.000000

	Cost Values (y_test)	Cost Values (y_pred)
0	14.659258	15.169384
1	10.282391	11.046716
2	13.712759	13.532547
3	13.562379	13.975668
4	13.551972	14.169768
5	10.360102	11.112930
6	10.282391	11.046716
7	14.659258	15.169384
8	13.338839	13.743884
9	12.731859	12.178673
10	58.727225	60.394838
11	55.486654	54.778530
12	26.552406	24.543799
13	33.430417	32.307067
14	25.659750	25.428151
15	32.054137	32.507028
16	21.545902	21.527582
17	32.054137	32.507028
18	10.282391	11.046716
19	13.716443	13.888073
20	32.162537	32.308385
21	13.712759	13.532547
22	32.061074	31.104951
23	14.874677	14.410994
24	16.390814	16.459691
25	14.659258	15.169384
26	10.282391	11.046716
27	21.782815	22.115788
28	8.396757	9.785264
29	8.218772	9.651939
30	157.818024	157.053589
31	10.282391	11.046716
32	16.354582	16.800889
33	10.282391	11.046716
34	9.014910	10.162423
35	13.712759	13.532547
36	33.430417	32.307067
37	28.939670	28.236510
38	13.716443	13.888073
39	13.338839	13.743884
40	10.335507	11.179706
41	10.282391	11.046716
42	32.162537	32.308385
43	18.422745	18.211289
44	32.054137	32.507028
45	14.659258	15.169384
46	12.731859	12.178673
47	24.677530	24.567735
48	11.885850	12.726585
49	14.029892	14.153191
50	13.712759	13.532547
51	10.718100	10.547687
52	32.054137	32.507028
53	14.029892	14.153191
54	9.014910	10.162423
55	12.731859	12.178673
56	14.659258	15.169384

	Cost Values MLP (y_test)	Cost Values MLP (y_pred)
0	14.659258	15.186515
1	10.282391	11.064986
2	13.712759	13.616386
3	13.562379	13.704834
4	13.551972	14.191711
5	10.360102	11.129772
6	10.282391	11.064986
7	14.659258	15.186515
8	13.338839	13.744403
9	12.731859	12.180622
10	58.727225	60.369253
11	55.486654	54.837077
12	26.552406	24.537801
13	33.430417	32.202356
14	25.659750	25.427744
15	32.054137	32.511438
16	21.545902	21.488325
17	32.054137	32.511438
18	10.282391	11.064986
19	13.716443	13.951159
20	32.162537	32.317081
21	13.712759	13.616386
22	32.061074	31.102882
23	14.874677	14.429175
24	16.390814	16.489562
25	14.659258	15.186515
26	10.282391	11.064986
27	21.782815	22.110400
28	8.396757	9.808152
29	8.218772	9.677743
30	157.818024	157.047723
31	10.282391	11.064986
32	16.354582	16.812044
33	10.282391	11.064986
34	9.014910	10.037760
35	13.712759	13.616386
36	33.430417	32.202356
37	28.939670	28.248670
38	13.716443	13.951159
39	13.338839	13.744403
40	10.335507	11.194702
41	10.282391	11.064986
42	32.162537	32.317081
43	18.422745	18.241106
44	32.054137	32.511438
45	14.659258	15.186515
46	12.731859	12.180622
47	24.677530	24.549556
48	11.885850	12.733645
49	14.029892	14.131553
50	13.712759	13.616386
51	10.718100	10.600549
52	32.054137	32.511438
53	14.029892	14.131553
54	9.014910	10.037760
55	12.731859	12.180622
56	14.659258	15.186515

Dep. Variable:	Yield Strength (MPa)	R-squared (uncentered):	0.860
Model:	OLS	Adj. R-squared (uncentered):	0.847
Method:	Least Squares	F-statistic:	69.45
Date:	Mon, 06 Sep 2021	Prob (F-statistic):	2.53e-97
Time:	20:55:39	Log-Likelihood:	-1949.5
No. Observations:	284	AIC:	3945.
Df Residuals:	261	BIC:	4029.
Df Model:	23
Covariance Type:	nonrobust

	coef	std err	t	P>\|t\|	[0.025	0.975]
Ti(at%)	5.7226	0.888	6.446	0.000	3.975	7.471
Mn(at%)	10.6805	10.919	0.978	0.329	-10.820	32.181
Nb(at%)	18.8432	2.728	6.908	0.000	13.472	24.215
Cr(at%)	104.8834	13.528	7.753	0.000	78.245	131.521
Mo(at%)	114.8525	33.592	3.419	0.001	48.707	180.998
V(at%)	31.3510	17.594	1.782	0.076	-3.292	65.994
Zr(at%)	53.0253	31.495	1.684	0.093	-8.991	115.042
Hf(at%)	60.7005	34.872	1.741	0.083	-7.965	129.366
Ta(at%)	48.7889	41.380	1.179	0.239	-32.693	130.271
Si(at%)	-242.0689	319.768	-0.757	0.450	-871.723	387.585
W(at%)	154.3153	82.924	1.861	0.064	-8.970	317.601
C(at%)	2.9836	26.369	0.113	0.910	-48.939	54.906
B(at%)	179.5952	50.976	3.523	0.001	79.219	279.971
Fe(at%)	68.1585	16.572	4.113	0.000	35.527	100.790
Ru(at%)	89.6692	40.417	2.219	0.027	10.085	169.253
Re(at%)	106.7288	580.208	0.184	0.854	-1035.756	1249.214
Ni(at%)	-49.7809	58.138	-0.856	0.393	-164.261	64.699
O(at%)	661.0055	480.023	1.377	0.170	-284.204	1606.215
Y(at%)	-292.4137	595.080	-0.491	0.624	-1464.183	879.356
TiB2(at%)	168.5829	82.342	2.047	0.042	6.444	330.721
Test Temperature (°C)	-0.1739	0.042	-4.155	0.000	-0.256	-0.091
Synthesis Methods	10.7582	16.050	0.670	0.503	-20.847	42.363
Microstructure	34.3601	19.104	1.799	0.073	-3.258	71.978

Omnibus:	29.627	Durbin-Watson:	1.147
Prob(Omnibus):	0.000	Jarque-Bera (JB):	51.554
Skew:	0.606	Prob(JB):	6.39e-12
Kurtosis:	4.700	Cond. No.	2.32e+04

Omnibus:	29.619	Durbin-Watson:	1.146
Prob(Omnibus):	0.000	Jarque-Bera (JB):	51.487
Skew:	0.606	Prob(JB):	6.60e-12
Kurtosis:	4.698	Cond. No.	2.32e+04

Omnibus:	29.585	Durbin-Watson:	1.146
Prob(Omnibus):	0.000	Jarque-Bera (JB):	51.464
Skew:	0.605	Prob(JB):	6.68e-12
Kurtosis:	4.699	Cond. No.	2.26e+04

Omnibus:	29.522	Durbin-Watson:	1.145
Prob(Omnibus):	0.000	Jarque-Bera (JB):	51.296
Skew:	0.604	Prob(JB):	7.26e-12
Kurtosis:	4.695	Cond. No.	1.79e+04

Omnibus:	27.794	Durbin-Watson:	1.139
Prob(Omnibus):	0.000	Jarque-Bera (JB):	47.162
Skew:	0.581	Prob(JB):	5.74e-11
Kurtosis:	4.623	Cond. No.	1.77e+04

Omnibus:	28.095	Durbin-Watson:	1.125
Prob(Omnibus):	0.000	Jarque-Bera (JB):	47.587
Skew:	0.587	Prob(JB):	4.64e-11
Kurtosis:	4.625	Cond. No.	1.77e+04

Omnibus:	27.160	Durbin-Watson:	1.116
Prob(Omnibus):	0.000	Jarque-Bera (JB):	45.049
Skew:	0.577	Prob(JB):	1.65e-10
Kurtosis:	4.573	Cond. No.	1.77e+04

Omnibus:	26.879	Durbin-Watson:	1.127
Prob(Omnibus):	0.000	Jarque-Bera (JB):	44.533
Skew:	0.573	Prob(JB):	2.14e-10
Kurtosis:	4.566	Cond. No.	1.77e+04

Omnibus:	27.083	Durbin-Watson:	1.125
Prob(Omnibus):	0.000	Jarque-Bera (JB):	44.816
Skew:	0.577	Prob(JB):	1.85e-10
Kurtosis:	4.567	Cond. No.	1.77e+04

Omnibus:	25.800	Durbin-Watson:	1.119
Prob(Omnibus):	0.000	Jarque-Bera (JB):	42.140
Skew:	0.557	Prob(JB):	7.07e-10
Kurtosis:	4.523	Cond. No.	1.76e+04

	Alloy composition (At %) Dec 2019 1228	Ti(at%)	Al(at%)	Mn(at%)	Nb(at%)	Cr(at%)	Mo(at%)	V(at%)	Zr(at%)	Hf(at%)	Ta(at%)	Si(at%)	W(at%)	C(at%)	B(at%)	Fe(at%)	Ru(at%)	Re(at%)	Ni(at%)	O(at%)	Y(at%)	TiB2(at%)	Test Temperature (°C)	Primary Creep (Elongation/s)	Secondary Creep (Elongation/s)	Lamaller Volume	Interlamellar Spacing (µm)	α Width (µm)	γ Width (µm)	γ Grain Size (µm)	α2-α2 Spacing (µm)	Vα	Vγ	Colony Size (µm)	Density (g/cm³)	Diamond Pyramid Hardness (DPH)	Vickers Hardness (HV)	Synthesis Methods	Conditions	Microstructure	Orientation (°)	Yield Strength (MPa)	Tensile Strength (MPa)	Elongation (%)	Log 10 (Elongation)	Fracture Toughness (MPa)
0	Ti-47Al	53.000	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	NaN	NaN	NaN	0.400	0.0900	0.425	NaN	0.520	0.30	0.70	NaN	NaN	307.0	NaN	casting	1 h at 1400°C Furnace cooled	lamellar	NaN	495.00	675.00	2.30	-1.638272	NaN
1	Ti-47Al-0.014B	52.986	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.014	NaN	NaN	NaN	NaN	NaN	NaN	NaN	600	NaN	NaN	NaN	0.170	0.7500	0.235	NaN	0.415	0.28	0.72	NaN	NaN	297.0	NaN	casting	1 h at 1400°C Furnace cooled	lamellar	NaN	502.00	690.00	5.60	-1.251812	NaN
2	Ti-47Al-0.5W-0.014B	52.486	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.5	NaN	0.014	NaN	NaN	NaN	NaN	NaN	NaN	NaN	750	NaN	NaN	NaN	0.190	0.0900	0.430	NaN	0.490	0.28	0.72	NaN	NaN	331.0	NaN	casting	1 h at 1400°C Furnace cooled	lamellar	NaN	450.00	587.00	3.50	-1.455932	NaN
3	Ti-47Al-0.014B	52.986	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.014	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	NaN	NaN	NaN	0.144	0.0925	0.375	NaN	0.370	0.28	0.72	NaN	NaN	305.0	NaN	casting	1 h at 1400°C Furnace cooled + aged 168 h/800°C	lamellar	NaN	550.00	679.00	2.70	-1.568636	NaN
4	Ti-47Al	53.000	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	750	NaN	NaN	NaN	1.375	0.6250	1.360	NaN	1.300	0.33	0.67	NaN	NaN	324.0	NaN	casting	1 h at 1400°C Furnace cooled+ aged 168 h/1000°C	lamellar	NaN	350.00	596.00	3.10	-1.508638	NaN
5	Ti-47Al-0.014B	52.986	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.014	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	NaN	NaN	NaN	0.375	0.1025	0.700	NaN	1.420	0.30	0.70	NaN	NaN	309.0	NaN	casting	1 h at 1400°C Furnace cooled+ aged 168 h/1000°C	lamellar	NaN	405.00	568.00	5.70	-1.244125	NaN
6	Ti-47Al-0.5W-0.014B	52.486	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.5	NaN	0.014	NaN	NaN	NaN	NaN	NaN	NaN	NaN	760	NaN	NaN	NaN	0.190	0.0950	0.295	NaN	0.660	0.29	0.71	NaN	NaN	344.0	NaN	casting	1 h at 1400°C Furnace cooled+ aged 168 h/1000°C	lamellar	NaN	604.00	790.00	1.20	-1.920819	NaN
7	Ti-47Al-0.014B	52.986	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.014	NaN	NaN	NaN	NaN	NaN	NaN	NaN	770	NaN	NaN	NaN	1.900	1.0000	3.000	NaN	3.700	0.32	0.68	NaN	NaN	293.0	NaN	casting	1 h at 1400°C Furnace cooled+ aged 168 h/1200°C	lamellar	NaN	450.00	687.00	1.70	-1.769551	NaN
8	Ti-45Al-2Mn-2Nb-0.8TiB2	50.200	45.0	2.0	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.80	25	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	casting	NaN	lamellar	NaN	570.00	695.00	1.50	-1.823909	17.0
9	Ti-45Al-2Mn-2Nb-0.8TiB2	50.200	45.0	2.0	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.80	600	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	casting	NaN	lamellar	NaN	440.00	650.00	5.70	-1.244125	NaN
10	Ti-45Al-2Mn-2Nb-0.8TiB2	50.200	45.0	2.0	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.80	760	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	casting	NaN	lamellar	NaN	415.00	510.00	19.00	-0.721246	NaN
11	Ti-47Al-2Mn-2Nb-0.8TiB2	48.200	47.0	2.0	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.80	25	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	casting	NaN	lamellar	NaN	402.00	482.00	1.50	-1.823909	15.5
12	Ti-47Al-2Mn-2Nb-0.8TiB2	48.200	47.0	2.0	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.80	760	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	casting	NaN	lamellar	NaN	344.00	458.00	7.90	-1.102373	NaN
13	Ti-47Al-2W-0.5Si	50.500	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.5	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	casting	NaN	duplex	NaN	425.00	520.00	1.00	-2.000000	22.0
14	Ti-47Al-2W-0.5Si	50.500	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.5	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	760	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	casting	NaN	duplex	NaN	350.00	460.00	2.50	-1.602060	NaN
15	Ti-46Al-4Nb-1W	49.000	46.0	NaN	4.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	extruding	NaN	lamellar	NaN	648.00	717.00	1.60	-1.795880	NaN
16	Ti-46Al-4Nb-1W	49.000	46.0	NaN	4.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	760	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	extruding	NaN	lamellar	NaN	517.00	692.00	5.40	-1.267606	NaN
17	Ti-46.5Al-2Cr-3Nb-0.2W	48.300	46.5	NaN	3.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	0.2	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	duplex	NaN	462.00	579.00	2.80	-1.552842	11.0
18	Ti-46.5Al-2Cr-3Nb-0.2W	48.300	46.5	NaN	3.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	0.2	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	800	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	duplex	NaN	345.00	468.00	40.00	-0.397940	NaN
19	Ti-46.5Al-2Cr-3Nb-0.2W	48.300	46.5	NaN	3.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	0.2	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	lamellar	NaN	473.00	557.00	1.20	-1.920819	21.0
20	Ti-46.5Al-2Cr-3Nb-0.2W	48.300	46.5	NaN	3.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	0.2	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	800	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	lamellar	NaN	375.00	509.00	3.20	-1.494850	NaN
21	Ti-46.5Al-2Cr-3Nb-0.2W	48.300	46.5	NaN	3.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	0.2	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	870	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	lamellar	NaN	362.00	485.00	12.00	-0.920819	NaN
22	Ti-48Al-1V-0.2C-0.14O	50.660	48.0	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	0.2	NaN	NaN	NaN	NaN	NaN	0.14	NaN	NaN	25	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	duplex	NaN	480.00	530.00	1.50	-1.823909	NaN
23	Ti-48Al-1V-0.2C-0.14O	50.660	48.0	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	0.2	NaN	NaN	NaN	NaN	NaN	0.14	NaN	NaN	815	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	duplex	NaN	360.00	450.00	4.20	-1.376751	NaN
24	Ti-48Al-1V-0.3C-0.2O	50.500	48.0	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	0.3	NaN	NaN	NaN	NaN	NaN	0.20	NaN	NaN	25	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	duplex	NaN	392.00	406.00	1.40	-1.853872	12.3
25	Ti-48Al-1V-0.3C-0.2O	50.500	48.0	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	0.3	NaN	NaN	NaN	NaN	NaN	0.20	NaN	NaN	437	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	duplex	NaN	345.00	476.00	6.40	-1.193820	22.8
26	Ti-48Al-1V-0.3C-0.2O	50.500	48.0	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	0.3	NaN	NaN	NaN	NaN	NaN	0.20	NaN	NaN	760	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	duplex	NaN	320.00	470.00	10.80	-0.966576	NaN
27	Ti-48Al-1V-0.3C-0.2O	50.500	48.0	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	0.3	NaN	NaN	NaN	NaN	NaN	0.20	NaN	NaN	25	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Casting	NaN	duplex	NaN	490.00	567.00	11.50	-0.939302	24.3
28	Ti-47Al-1Cr-1V-2.6Nb	48.400	47.0	NaN	2.6	1.00	NaN	1.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	duplex	NaN	480.00	548.00	2.30	-1.638272	12.0
29	Ti-47Al-1Cr-1V-2.6Nb	48.400	47.0	NaN	2.6	1.00	NaN	1.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	600	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	duplex	NaN	383.00	507.00	3.10	-1.508638	16.0
30	Ti-47Al-1Cr-1V-2.6Nb	48.400	47.0	NaN	2.6	1.00	NaN	1.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	800	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	duplex	NaN	324.00	492.00	55.00	-0.259637	NaN
31	Ti-47Al-1Cr-1V-2.6Nb	48.400	47.0	NaN	2.6	1.00	NaN	1.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	lamellar	NaN	330.00	383.00	0.80	-2.096910	33.0
32	Ti-47Al-1Cr-1V-2.6Nb	48.400	47.0	NaN	2.6	1.00	NaN	1.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	800	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	forging	NaN	lamellar	NaN	290.00	378.00	1.50	-1.823909	55.0
33	Ti-45Al-16Mn	39.000	45.0	16.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	sintering	NaN	lamellar	NaN	465.00	566.00	1.40	-1.853872	NaN
34	Ti-45Al-16Mn	39.000	45.0	16.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	800	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	sintering	NaN	lamellar	NaN	370.00	540.00	14.00	-0.853872	NaN
35	Ti-48Al-2Cr-2Nb	48.000	48.0	NaN	2.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	casting	NaN	duplex	NaN	331.00	413.00	2.30	-1.638272	25.0
36	Ti-48Al-2Cr-2Nb	48.000	48.0	NaN	2.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	760	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	casting	NaN	duplex	NaN	310.00	430.00	6.40	-1.193820	NaN
37	Ti-48Al-2Cr-2Nb	48.000	48.0	NaN	2.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	extruding	NaN	duplex	NaN	480.00	679.00	3.10	-1.508638	NaN
38	Ti-48Al-2Cr-2Nb	48.000	48.0	NaN	2.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	760	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	extruding	NaN	duplex	NaN	403.00	563.00	40.00	-0.397940	NaN
39	Ti-48Al-2Cr-2Nb	48.000	48.0	NaN	2.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	870	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	extruding	NaN	duplex	NaN	330.00	475.00	53.00	-0.275724	NaN

	Ti(at%)	Al(at%)	Mn(at%)	Nb(at%)	Cr(at%)	Mo(at%)	V(at%)	Zr(at%)	Hf(at%)	Ta(at%)	Si(at%)	W(at%)	C(at%)	B(at%)	Fe(at%)	Ru(at%)	Re(at%)	Ni(at%)	O(at%)	Y(at%)	TiB2(at%)	Test Temperature (°C)	Synthesis Methods	Microstructure	Yield Strength (MPa)	Tensile Strength (MPa)	Log 10 (Elongation)
0	53.000	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	casting	lamellar	495.00	675.00	-1.638272
1	52.986	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.014	NaN	NaN	NaN	NaN	NaN	NaN	NaN	600	casting	lamellar	502.00	690.00	-1.251812
2	52.486	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.5	NaN	0.014	NaN	NaN	NaN	NaN	NaN	NaN	NaN	750	casting	lamellar	450.00	587.00	-1.455932
3	52.986	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.014	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	casting	lamellar	550.00	679.00	-1.568636
4	53.000	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	750	casting	lamellar	350.00	596.00	-1.508638
5	52.986	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.014	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	casting	lamellar	405.00	568.00	-1.244125
6	52.486	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.5	NaN	0.014	NaN	NaN	NaN	NaN	NaN	NaN	NaN	760	casting	lamellar	604.00	790.00	-1.920819
7	52.986	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.014	NaN	NaN	NaN	NaN	NaN	NaN	NaN	770	casting	lamellar	450.00	687.00	-1.769551
8	50.200	45.0	2.0	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.80	25	casting	lamellar	570.00	695.00	-1.823909
9	50.200	45.0	2.0	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.80	600	casting	lamellar	440.00	650.00	-1.244125
10	50.200	45.0	2.0	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.80	760	casting	lamellar	415.00	510.00	-0.721246
11	48.200	47.0	2.0	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.80	25	casting	lamellar	402.00	482.00	-1.823909
12	48.200	47.0	2.0	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.80	760	casting	lamellar	344.00	458.00	-1.102373
13	50.500	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.5	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	casting	duplex	425.00	520.00	-2.000000
14	50.500	47.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	0.5	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	760	casting	duplex	350.00	460.00	-1.602060
15	49.000	46.0	NaN	4.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	extruding	lamellar	648.00	717.00	-1.795880
16	49.000	46.0	NaN	4.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	760	extruding	lamellar	517.00	692.00	-1.267606
17	48.300	46.5	NaN	3.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	0.2	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	forging	duplex	462.00	579.00	-1.552842
18	48.300	46.5	NaN	3.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	0.2	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	800	forging	duplex	345.00	468.00	-0.397940
19	48.300	46.5	NaN	3.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	0.2	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	forging	lamellar	473.00	557.00	-1.920819
20	48.300	46.5	NaN	3.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	0.2	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	800	forging	lamellar	375.00	509.00	-1.494850
21	48.300	46.5	NaN	3.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	0.2	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	870	forging	lamellar	362.00	485.00	-0.920819
22	50.660	48.0	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	0.2	NaN	NaN	NaN	NaN	NaN	0.14	NaN	NaN	25	forging	duplex	480.00	530.00	-1.823909
23	50.660	48.0	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	0.2	NaN	NaN	NaN	NaN	NaN	0.14	NaN	NaN	815	forging	duplex	360.00	450.00	-1.376751
24	50.500	48.0	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	0.3	NaN	NaN	NaN	NaN	NaN	0.20	NaN	NaN	25	forging	duplex	392.00	406.00	-1.853872
25	50.500	48.0	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	0.3	NaN	NaN	NaN	NaN	NaN	0.20	NaN	NaN	437	forging	duplex	345.00	476.00	-1.193820
26	50.500	48.0	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	0.3	NaN	NaN	NaN	NaN	NaN	0.20	NaN	NaN	760	forging	duplex	320.00	470.00	-0.966576
27	50.500	48.0	NaN	NaN	NaN	NaN	1.0	NaN	NaN	NaN	NaN	NaN	0.3	NaN	NaN	NaN	NaN	NaN	0.20	NaN	NaN	25	Casting	duplex	490.00	567.00	-0.939302
28	48.400	47.0	NaN	2.6	1.00	NaN	1.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	forging	duplex	480.00	548.00	-1.638272
29	48.400	47.0	NaN	2.6	1.00	NaN	1.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	600	forging	duplex	383.00	507.00	-1.508638
30	48.400	47.0	NaN	2.6	1.00	NaN	1.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	800	forging	duplex	324.00	492.00	-0.259637
31	48.400	47.0	NaN	2.6	1.00	NaN	1.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	forging	lamellar	330.00	383.00	-2.096910
32	48.400	47.0	NaN	2.6	1.00	NaN	1.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	800	forging	lamellar	290.00	378.00	-1.823909
33	39.000	45.0	16.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	sintering	lamellar	465.00	566.00	-1.853872
34	39.000	45.0	16.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	800	sintering	lamellar	370.00	540.00	-0.853872
35	48.000	48.0	NaN	2.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	casting	duplex	331.00	413.00	-1.638272
36	48.000	48.0	NaN	2.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	760	casting	duplex	310.00	430.00	-1.193820
37	48.000	48.0	NaN	2.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	25	extruding	duplex	480.00	679.00	-1.508638
38	48.000	48.0	NaN	2.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	760	extruding	duplex	403.00	563.00	-0.397940
39	48.000	48.0	NaN	2.0	2.00	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	870	extruding	duplex	330.00	475.00	-0.275724

	Ti(at%)	Al(at%)	Mn(at%)	Nb(at%)	Cr(at%)	V(at%)	Si(at%)	W(at%)	C(at%)	B(at%)	O(at%)	TiB2(at%)	Test Temperature (°C)	Synthesis Methods	Microstructure	Yield Strength (MPa)	Tensile Strength (MPa)	Log 10 (Elongation)
0	53.000	47.0	0.0	0.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	25	casting	lamellar	495.00	675.00	-1.638272
1	52.986	47.0	0.0	0.0	0.00	0.0	0.0	0.0	0.0	0.014	0.00	0.00	600	casting	lamellar	502.00	690.00	-1.251812
2	52.486	47.0	0.0	0.0	0.00	0.0	0.0	0.5	0.0	0.014	0.00	0.00	750	casting	lamellar	450.00	587.00	-1.455932
3	52.986	47.0	0.0	0.0	0.00	0.0	0.0	0.0	0.0	0.014	0.00	0.00	25	casting	lamellar	550.00	679.00	-1.568636
4	53.000	47.0	0.0	0.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	750	casting	lamellar	350.00	596.00	-1.508638
5	52.986	47.0	0.0	0.0	0.00	0.0	0.0	0.0	0.0	0.014	0.00	0.00	25	casting	lamellar	405.00	568.00	-1.244125
6	52.486	47.0	0.0	0.0	0.00	0.0	0.0	0.5	0.0	0.014	0.00	0.00	760	casting	lamellar	604.00	790.00	-1.920819
7	52.986	47.0	0.0	0.0	0.00	0.0	0.0	0.0	0.0	0.014	0.00	0.00	770	casting	lamellar	450.00	687.00	-1.769551
8	50.200	45.0	2.0	2.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.80	25	casting	lamellar	570.00	695.00	-1.823909
9	50.200	45.0	2.0	2.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.80	600	casting	lamellar	440.00	650.00	-1.244125
10	50.200	45.0	2.0	2.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.80	760	casting	lamellar	415.00	510.00	-0.721246
11	48.200	47.0	2.0	2.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.80	25	casting	lamellar	402.00	482.00	-1.823909
12	48.200	47.0	2.0	2.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.80	760	casting	lamellar	344.00	458.00	-1.102373
13	50.500	47.0	0.0	0.0	0.00	0.0	0.5	2.0	0.0	0.000	0.00	0.00	25	casting	duplex	425.00	520.00	-2.000000
14	50.500	47.0	0.0	0.0	0.00	0.0	0.5	2.0	0.0	0.000	0.00	0.00	760	casting	duplex	350.00	460.00	-1.602060
15	49.000	46.0	0.0	4.0	0.00	0.0	0.0	1.0	0.0	0.000	0.00	0.00	25	extruding	lamellar	648.00	717.00	-1.795880
16	49.000	46.0	0.0	4.0	0.00	0.0	0.0	1.0	0.0	0.000	0.00	0.00	760	extruding	lamellar	517.00	692.00	-1.267606
17	48.300	46.5	0.0	3.0	2.00	0.0	0.0	0.2	0.0	0.000	0.00	0.00	25	forging	duplex	462.00	579.00	-1.552842
18	48.300	46.5	0.0	3.0	2.00	0.0	0.0	0.2	0.0	0.000	0.00	0.00	800	forging	duplex	345.00	468.00	-0.397940
19	48.300	46.5	0.0	3.0	2.00	0.0	0.0	0.2	0.0	0.000	0.00	0.00	25	forging	lamellar	473.00	557.00	-1.920819
20	48.300	46.5	0.0	3.0	2.00	0.0	0.0	0.2	0.0	0.000	0.00	0.00	800	forging	lamellar	375.00	509.00	-1.494850
21	48.300	46.5	0.0	3.0	2.00	0.0	0.0	0.2	0.0	0.000	0.00	0.00	870	forging	lamellar	362.00	485.00	-0.920819
22	50.660	48.0	0.0	0.0	0.00	1.0	0.0	0.0	0.2	0.000	0.14	0.00	25	forging	duplex	480.00	530.00	-1.823909
23	50.660	48.0	0.0	0.0	0.00	1.0	0.0	0.0	0.2	0.000	0.14	0.00	815	forging	duplex	360.00	450.00	-1.376751
24	50.500	48.0	0.0	0.0	0.00	1.0	0.0	0.0	0.3	0.000	0.20	0.00	25	forging	duplex	392.00	406.00	-1.853872
25	50.500	48.0	0.0	0.0	0.00	1.0	0.0	0.0	0.3	0.000	0.20	0.00	437	forging	duplex	345.00	476.00	-1.193820
26	50.500	48.0	0.0	0.0	0.00	1.0	0.0	0.0	0.3	0.000	0.20	0.00	760	forging	duplex	320.00	470.00	-0.966576
27	50.500	48.0	0.0	0.0	0.00	1.0	0.0	0.0	0.3	0.000	0.20	0.00	25	Casting	duplex	490.00	567.00	-0.939302
28	48.400	47.0	0.0	2.6	1.00	1.0	0.0	0.0	0.0	0.000	0.00	0.00	25	forging	duplex	480.00	548.00	-1.638272
29	48.400	47.0	0.0	2.6	1.00	1.0	0.0	0.0	0.0	0.000	0.00	0.00	600	forging	duplex	383.00	507.00	-1.508638
30	48.400	47.0	0.0	2.6	1.00	1.0	0.0	0.0	0.0	0.000	0.00	0.00	800	forging	duplex	324.00	492.00	-0.259637
31	48.400	47.0	0.0	2.6	1.00	1.0	0.0	0.0	0.0	0.000	0.00	0.00	25	forging	lamellar	330.00	383.00	-2.096910
32	48.400	47.0	0.0	2.6	1.00	1.0	0.0	0.0	0.0	0.000	0.00	0.00	800	forging	lamellar	290.00	378.00	-1.823909
33	39.000	45.0	16.0	0.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	25	sintering	lamellar	465.00	566.00	-1.853872
34	39.000	45.0	16.0	0.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	800	sintering	lamellar	370.00	540.00	-0.853872
35	48.000	48.0	0.0	2.0	2.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	25	casting	duplex	331.00	413.00	-1.638272
36	48.000	48.0	0.0	2.0	2.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	760	casting	duplex	310.00	430.00	-1.193820
37	48.000	48.0	0.0	2.0	2.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	25	extruding	duplex	480.00	679.00	-1.508638
38	48.000	48.0	0.0	2.0	2.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	760	extruding	duplex	403.00	563.00	-0.397940
39	48.000	48.0	0.0	2.0	2.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	870	extruding	duplex	330.00	475.00	-0.275724

	Ti(at%)	Al(at%)	Mn(at%)	Nb(at%)	Cr(at%)	V(at%)	Si(at%)	W(at%)	C(at%)	B(at%)	O(at%)	TiB2(at%)	Test Temperature (°C)	Synthesis Methods	Microstructure	Yield Strength (MPa)	Tensile Strength (MPa)	Log 10 (Elongation)	Ti(wt%)	Al(wt%)	Mn(wt%)	Nb(wt%)	Cr(wt%)	V(wt%)	Si(wt%)	W(wt%)	C(wt%)	B(wt%)	O(wt%)	TiB2(wt%)
0	53.000	47.0	0.0	0.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	25	casting	lamellar	495.00	675.00	-1.638272	0.666753	0.333247	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
1	52.986	47.0	0.0	0.0	0.00	0.0	0.0	0.0	0.0	0.014	0.00	0.00	600	casting	lamellar	502.00	690.00	-1.251812	0.666668	0.333292	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000040	0.000000	0.000000
2	52.486	47.0	0.0	0.0	0.00	0.0	0.0	0.5	0.0	0.014	0.00	0.00	750	casting	lamellar	450.00	587.00	-1.455932	0.648784	0.327441	0.000000	0.000000	0.000000	0.000000	0.000000	0.023736	0.000000	0.000039	0.000000	0.000000
3	52.986	47.0	0.0	0.0	0.00	0.0	0.0	0.0	0.0	0.014	0.00	0.00	25	casting	lamellar	550.00	679.00	-1.568636	0.666668	0.333292	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000040	0.000000	0.000000
4	53.000	47.0	0.0	0.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	750	casting	lamellar	350.00	596.00	-1.508638	0.666753	0.333247	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
5	52.986	47.0	0.0	0.0	0.00	0.0	0.0	0.0	0.0	0.014	0.00	0.00	25	casting	lamellar	405.00	568.00	-1.244125	0.666668	0.333292	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000040	0.000000	0.000000
6	52.486	47.0	0.0	0.0	0.00	0.0	0.0	0.5	0.0	0.014	0.00	0.00	760	casting	lamellar	604.00	790.00	-1.920819	0.648784	0.327441	0.000000	0.000000	0.000000	0.000000	0.000000	0.023736	0.000000	0.000039	0.000000	0.000000
7	52.986	47.0	0.0	0.0	0.00	0.0	0.0	0.0	0.0	0.014	0.00	0.00	770	casting	lamellar	450.00	687.00	-1.769551	0.666668	0.333292	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000040	0.000000	0.000000
8	50.200	45.0	2.0	2.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.80	25	casting	lamellar	570.00	695.00	-1.823909	0.618291	0.312378	0.028271	0.026756	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.014303
9	50.200	45.0	2.0	2.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.80	600	casting	lamellar	440.00	650.00	-1.244125	0.618291	0.312378	0.028271	0.026756	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.014303
10	50.200	45.0	2.0	2.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.80	760	casting	lamellar	415.00	510.00	-0.721246	0.618291	0.312378	0.028271	0.026756	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.014303
11	48.200	47.0	2.0	2.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.80	25	casting	lamellar	402.00	482.00	-1.823909	0.600109	0.329807	0.028578	0.027047	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.014459
12	48.200	47.0	2.0	2.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.80	760	casting	lamellar	344.00	458.00	-1.102373	0.600109	0.329807	0.028578	0.027047	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.014459
13	50.500	47.0	0.0	0.0	0.00	0.0	0.5	2.0	0.0	0.000	0.00	0.00	25	casting	duplex	425.00	520.00	-2.000000	0.594955	0.312083	0.000000	0.000000	0.000000	0.000000	0.002472	0.090490	0.000000	0.000000	0.000000	0.000000
14	50.500	47.0	0.0	0.0	0.00	0.0	0.5	2.0	0.0	0.000	0.00	0.00	760	casting	duplex	350.00	460.00	-1.602060	0.594955	0.312083	0.000000	0.000000	0.000000	0.000000	0.002472	0.090490	0.000000	0.000000	0.000000	0.000000
15	49.000	46.0	0.0	4.0	0.00	0.0	0.0	1.0	0.0	0.000	0.00	0.00	25	extruding	lamellar	648.00	717.00	-1.795880	0.589571	0.311944	0.000000	0.052277	0.000000	0.000000	0.000000	0.046208	0.000000	0.000000	0.000000	0.000000
16	49.000	46.0	0.0	4.0	0.00	0.0	0.0	1.0	0.0	0.000	0.00	0.00	760	extruding	lamellar	517.00	692.00	-1.267606	0.589571	0.311944	0.000000	0.052277	0.000000	0.000000	0.000000	0.046208	0.000000	0.000000	0.000000	0.000000
17	48.300	46.5	0.0	3.0	2.00	0.0	0.0	0.2	0.0	0.000	0.00	0.00	25	forging	duplex	462.00	579.00	-1.552842	0.598461	0.324728	0.000000	0.040375	0.026919	0.000000	0.000000	0.009517	0.000000	0.000000	0.000000	0.000000
18	48.300	46.5	0.0	3.0	2.00	0.0	0.0	0.2	0.0	0.000	0.00	0.00	800	forging	duplex	345.00	468.00	-0.397940	0.598461	0.324728	0.000000	0.040375	0.026919	0.000000	0.000000	0.009517	0.000000	0.000000	0.000000	0.000000
19	48.300	46.5	0.0	3.0	2.00	0.0	0.0	0.2	0.0	0.000	0.00	0.00	25	forging	lamellar	473.00	557.00	-1.920819	0.598461	0.324728	0.000000	0.040375	0.026919	0.000000	0.000000	0.009517	0.000000	0.000000	0.000000	0.000000
20	48.300	46.5	0.0	3.0	2.00	0.0	0.0	0.2	0.0	0.000	0.00	0.00	800	forging	lamellar	375.00	509.00	-1.494850	0.598461	0.324728	0.000000	0.040375	0.026919	0.000000	0.000000	0.009517	0.000000	0.000000	0.000000	0.000000
21	48.300	46.5	0.0	3.0	2.00	0.0	0.0	0.2	0.0	0.000	0.00	0.00	870	forging	lamellar	362.00	485.00	-0.920819	0.598461	0.324728	0.000000	0.040375	0.026919	0.000000	0.000000	0.009517	0.000000	0.000000	0.000000	0.000000
22	50.660	48.0	0.0	0.0	0.00	1.0	0.0	0.0	0.2	0.000	0.14	0.00	25	forging	duplex	480.00	530.00	-1.823909	0.641607	0.342629	0.000000	0.000000	0.000000	0.014535	0.000000	0.000000	0.000635	0.000000	0.000593	0.000000
23	50.660	48.0	0.0	0.0	0.00	1.0	0.0	0.0	0.2	0.000	0.14	0.00	815	forging	duplex	360.00	450.00	-1.376751	0.641607	0.342629	0.000000	0.000000	0.000000	0.014535	0.000000	0.000000	0.000635	0.000000	0.000593	0.000000
24	50.500	48.0	0.0	0.0	0.00	1.0	0.0	0.0	0.3	0.000	0.20	0.00	25	forging	duplex	392.00	406.00	-1.853872	0.640513	0.343128	0.000000	0.000000	0.000000	0.014557	0.000000	0.000000	0.000955	0.000000	0.000848	0.000000
25	50.500	48.0	0.0	0.0	0.00	1.0	0.0	0.0	0.3	0.000	0.20	0.00	437	forging	duplex	345.00	476.00	-1.193820	0.640513	0.343128	0.000000	0.000000	0.000000	0.014557	0.000000	0.000000	0.000955	0.000000	0.000848	0.000000
26	50.500	48.0	0.0	0.0	0.00	1.0	0.0	0.0	0.3	0.000	0.20	0.00	760	forging	duplex	320.00	470.00	-0.966576	0.640513	0.343128	0.000000	0.000000	0.000000	0.014557	0.000000	0.000000	0.000955	0.000000	0.000848	0.000000
27	50.500	48.0	0.0	0.0	0.00	1.0	0.0	0.0	0.3	0.000	0.20	0.00	25	Casting	duplex	490.00	567.00	-0.939302	0.640513	0.343128	0.000000	0.000000	0.000000	0.014557	0.000000	0.000000	0.000955	0.000000	0.000848	0.000000
28	48.400	47.0	0.0	2.6	1.00	1.0	0.0	0.0	0.0	0.000	0.00	0.00	25	forging	duplex	480.00	548.00	-1.638272	0.605396	0.331337	0.000000	0.035324	0.013587	0.014356	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
29	48.400	47.0	0.0	2.6	1.00	1.0	0.0	0.0	0.0	0.000	0.00	0.00	600	forging	duplex	383.00	507.00	-1.508638	0.605396	0.331337	0.000000	0.035324	0.013587	0.014356	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
30	48.400	47.0	0.0	2.6	1.00	1.0	0.0	0.0	0.0	0.000	0.00	0.00	800	forging	duplex	324.00	492.00	-0.259637	0.605396	0.331337	0.000000	0.035324	0.013587	0.014356	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
31	48.400	47.0	0.0	2.6	1.00	1.0	0.0	0.0	0.0	0.000	0.00	0.00	25	forging	lamellar	330.00	383.00	-2.096910	0.605396	0.331337	0.000000	0.035324	0.013587	0.014356	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
32	48.400	47.0	0.0	2.6	1.00	1.0	0.0	0.0	0.0	0.000	0.00	0.00	800	forging	lamellar	290.00	378.00	-1.823909	0.605396	0.331337	0.000000	0.035324	0.013587	0.014356	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
33	39.000	45.0	16.0	0.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	25	sintering	lamellar	465.00	566.00	-1.853872	0.471439	0.306585	0.221976	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
34	39.000	45.0	16.0	0.0	0.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	800	sintering	lamellar	370.00	540.00	-0.853872	0.471439	0.306585	0.221976	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
35	48.000	48.0	0.0	2.0	2.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	25	casting	duplex	331.00	413.00	-1.638272	0.604548	0.340729	0.000000	0.027361	0.027363	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
36	48.000	48.0	0.0	2.0	2.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	760	casting	duplex	310.00	430.00	-1.193820	0.604548	0.340729	0.000000	0.027361	0.027363	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
37	48.000	48.0	0.0	2.0	2.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	25	extruding	duplex	480.00	679.00	-1.508638	0.604548	0.340729	0.000000	0.027361	0.027363	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
38	48.000	48.0	0.0	2.0	2.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	760	extruding	duplex	403.00	563.00	-0.397940	0.604548	0.340729	0.000000	0.027361	0.027363	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
39	48.000	48.0	0.0	2.0	2.00	0.0	0.0	0.0	0.0	0.000	0.00	0.00	870	extruding	duplex	330.00	475.00	-0.275724	0.604548	0.340729	0.000000	0.027361	0.027363	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000

Omnibus:	26.284	Durbin-Watson:	1.098
Prob(Omnibus):	0.000	Jarque-Bera (JB):	43.882
Skew:	0.558	Prob(JB):	2.96e-10
Kurtosis:	4.569	Cond. No.	3.11e+03

Omnibus:	26.296	Durbin-Watson:	1.067
Prob(Omnibus):	0.000	Jarque-Bera (JB):	43.715
Skew:	0.560	Prob(JB):	3.22e-10
Kurtosis:	4.562	Cond. No.	3.08e+03

Omnibus:	26.453	Durbin-Watson:	1.067
Prob(Omnibus):	0.000	Jarque-Bera (JB):	44.908
Skew:	0.555	Prob(JB):	1.77e-10
Kurtosis:	4.600	Cond. No.	3.07e+03

Omnibus:	27.295	Durbin-Watson:	1.067
Prob(Omnibus):	0.000	Jarque-Bera (JB):	46.590
Skew:	0.569	Prob(JB):	7.64e-11
Kurtosis:	4.625	Cond. No.	3.06e+03

Omnibus:	25.930	Durbin-Watson:	1.047
Prob(Omnibus):	0.000	Jarque-Bera (JB):	43.155
Skew:	0.553	Prob(JB):	4.26e-10
Kurtosis:	4.557	Cond. No.	3.06e+03

Omnibus:	27.935	Durbin-Watson:	1.025
Prob(Omnibus):	0.000	Jarque-Bera (JB):	47.122
Skew:	0.586	Prob(JB):	5.86e-11
Kurtosis:	4.615	Cond. No.	3.05e+03

Omnibus:	27.504	Durbin-Watson:	1.002
Prob(Omnibus):	0.000	Jarque-Bera (JB):	45.160
Skew:	0.588	Prob(JB):	1.56e-10
Kurtosis:	4.560	Cond. No.	1.72e+03

	Yield Strength Values MLP (y_test)	Yield Strength Values MLP (y_pred)
0	310.00	499.474536
1	570.00	553.325038
2	720.00	354.905914
3	305.00	597.568600
4	360.00	448.732178
5	353.31	669.811356
6	681.00	495.141192
7	567.00	515.850358
8	1560.00	1350.767152
9	367.00	389.319554
10	420.00	288.740506
11	940.00	921.501317
12	943.00	609.930689
13	320.00	415.689648
14	100.00	371.585276
15	342.00	410.292167
16	375.00	339.559025
17	458.00	511.586919
18	497.00	456.143839
19	440.00	746.238149
20	342.00	498.258354
21	368.00	354.509785
22	370.00	316.621009
23	617.00	911.757892
24	680.00	342.728145
25	457.00	498.258354
26	300.00	398.582638
27	413.00	441.249366
28	290.00	411.181742
29	517.00	458.189964
30	383.00	443.594409
31	460.00	646.640715
32	503.00	596.889968
33	586.21	669.811356
34	470.00	442.845643
35	547.00	470.246545
36	341.45	489.941533
37	425.00	401.865331
38	962.00	553.152839
39	1653.00	1223.698410
40	763.00	770.528665
41	490.00	627.102854
42	340.00	329.344754
43	305.00	398.567638
44	430.00	287.102323
45	733.00	911.757892
46	1917.00	1682.833639
47	579.00	503.017864
48	676.00	580.690717
49	607.00	597.941152
50	500.00	493.315420
51	358.00	550.787240
52	710.00	580.690717
53	480.00	501.390942
54	340.00	381.061908
55	624.00	731.180948
56	330.00	329.344754

Omnibus:	32.613	Durbin-Watson:	1.261
Prob(Omnibus):	0.000	Jarque-Bera (JB):	53.400
Skew:	0.684	Prob(JB):	2.54e-12
Kurtosis:	4.625	Cond. No.	2.32e+04

Omnibus:	32.530	Durbin-Watson:	1.260
Prob(Omnibus):	0.000	Jarque-Bera (JB):	53.270
Skew:	0.682	Prob(JB):	2.71e-12
Kurtosis:	4.624	Cond. No.	2.27e+04

Omnibus:	32.629	Durbin-Watson:	1.260
Prob(Omnibus):	0.000	Jarque-Bera (JB):	53.535
Skew:	0.683	Prob(JB):	2.37e-12
Kurtosis:	4.630	Cond. No.	2.27e+04

Omnibus:	34.211	Durbin-Watson:	1.263
Prob(Omnibus):	0.000	Jarque-Bera (JB):	57.598
Skew:	0.702	Prob(JB):	3.11e-13
Kurtosis:	4.702	Cond. No.	2.26e+04

Omnibus:	33.963	Durbin-Watson:	1.262
Prob(Omnibus):	0.000	Jarque-Bera (JB):	56.842
Skew:	0.700	Prob(JB):	4.54e-13
Kurtosis:	4.687	Cond. No.	1.78e+04

Omnibus:	33.058	Durbin-Watson:	1.263
Prob(Omnibus):	0.000	Jarque-Bera (JB):	54.669
Skew:	0.688	Prob(JB):	1.35e-12
Kurtosis:	4.651	Cond. No.	1.78e+04

Omnibus:	33.256	Durbin-Watson:	1.263
Prob(Omnibus):	0.000	Jarque-Bera (JB):	55.606
Skew:	0.687	Prob(JB):	8.42e-13
Kurtosis:	4.676	Cond. No.	1.78e+04

Omnibus:	32.768	Durbin-Watson:	1.273
Prob(Omnibus):	0.000	Jarque-Bera (JB):	54.359
Skew:	0.682	Prob(JB):	1.57e-12
Kurtosis:	4.654	Cond. No.	1.78e+04

Omnibus:	32.433	Durbin-Watson:	1.252
Prob(Omnibus):	0.000	Jarque-Bera (JB):	53.124
Skew:	0.681	Prob(JB):	2.91e-12
Kurtosis:	4.624	Cond. No.	1.78e+04

Omnibus:	32.157	Durbin-Watson:	1.257
Prob(Omnibus):	0.000	Jarque-Bera (JB):	51.982
Skew:	0.681	Prob(JB):	5.15e-12
Kurtosis:	4.593	Cond. No.	1.76e+04

Omnibus:	30.348	Durbin-Watson:	1.243
Prob(Omnibus):	0.000	Jarque-Bera (JB):	47.424
Skew:	0.661	Prob(JB):	5.04e-11
Kurtosis:	4.504	Cond. No.	1.76e+04

Omnibus:	29.802	Durbin-Watson:	1.222
Prob(Omnibus):	0.000	Jarque-Bera (JB):	45.880
Skew:	0.657	Prob(JB):	1.09e-10
Kurtosis:	4.467	Cond. No.	1.76e+04